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Preface 


The  events  of  9/11  shifted  the  baseline  of  life  in  the  United  States  and  beyond.  As  with  the  1941 
assault  on  Pearl  Harbor,  they  unmoored  perceptions  from  long-held  assumptions  and  entire 
populations  struggled  to  find  new  ways  to  understand  the  world.  Many  publications  describe  and 
document  how  these  events  and  subsequent  conflicts  impacted  the  requirements  of  the  U.S. 
national  security  enterprise.  Although  many  of  those  requirements  were  for  conventional 
approaches  to  solving  new-found  problems,  the  attacks  also  prompted  an  increased  call  for  a 
sociocultural  capability  to  understand  novel  cultural  environments,  anticipate  perceptions  and 
behaviors,  and  identify  alternative  approaches  to  influencing  those  behaviors.  While  the  merit  of 
such  a  capability  for  national  defense  has  long  been  recognized,  9/11  and  the  conflicts  that 
followed  sharpened  the  appreciation  of  its  value  and  accelerated  both  investment  and  innovation. 

The  U.S.  national  security  enterprise  adapted  its  research  agenda  to  address  the  increased  demand 
for  improved  sociocultural  behavior  capability.  The  largest  single  initiative,  led  by  the  Office  of  the 
Secretary  of  Defense,  was  the  Human  Social  Culture  Behavior  (HSCB)  Modeling  Program.  Objectives 
of  this  six-year  (2008-2013)  program  were  derived  from  key  strategic  documents,  such  as  the  2006 
Quadrennial  Defense  Review  and  the  subsequent  Strategic  Planning  Guidance  (SPG)  (2008-13) 
study  on  human  social  cultural  behavior  modeling.  Objectives  were  also  shaped  by  further  review 
of  sociocultural  behavior  research  efforts  across  the  Department  of  Defense  (DoD)  and  dialogue 
with  representatives  from  both  research  and  end-user  communities. 

One  touchstone  was  the  2008  National  Academies  Press  publication  Behavioral  Modeling  and 
Simulation:  From  Individuals  to  Societies.  That  report  served  as  a  foundation  for  the  HSCB  Modeling 
Program  Team  and  the  community  as  a  whole.  It  provided  model  developers,  operational  military 
users  of  the  models  and  their  managers,  and  government  personnel  making  funding  decisions 
regarding  model  development  a  timely  snapshot  of  the  field  prior  to  the  DoD  research  investments. 

Sociocultural  Behavior  Sensemaking:  State  of  the  Art  in  Understanding  the  Operational 
Environment  provides  another  snapshot,  taken  at  the  end  of  the  DoD's  HSCB  Modeling  Program's 
research  investment,  a  time  of  numerous  advances  in  the  area  of  sociocultural  behavior  analysis 
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and  modeling.  Although  the  domain  is  still  nascent  and  must  continue  to  evolve,  this  edited  volume 
captures  the  state  of  the  art  following  the  first  decade  of  change. 

One  of  the  core  goals  of  the  HSCB  program  was  to  develop  models  and  model-based  tools  that 
could  be  transitioned  to  programs  for  use  by  analysts,  planners,  and  operators  across  the  U.S. 
government.  The  present  book  shows  the  significant  progress  in  closing  the  research  gaps 
documented  in  the  2006  SPG  study  and  the  advances  made  since  the  publication  of  Behavioral 
Modeling  and  Simulation:  From  Individuals  to  Societies.  Each  chapter  of  the  present  book  describes 
leading-edge  research  and  development  in  this  domain,  which,  with  some  further  DoD 
investments,  could  also  be  transitioned  to  active  use.  Such  use  would  increase  our  warfighters' 
ability  to  leverage  a  sociocultural  behavior  capability  in  their  mission  planning  and  operations. 


Dylan  Schmorrow  John  Boiney 

Captain  Dylan  Schmorrow,  PhD  John  Boiney,  PhD 
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Introduction  to  Sociocultural  Sensemaking 

Gary  L.  Klein,  The  MITRE  Corporation 


Sociocultural  behavior  goes  beyond  mere  cultural  customs  and  rituals.  It  includes  the  common 
ways  that  a  group  of  people  interpret  cause  and  effect  and  the  mental  models  of  the  world  that 
they  share.  Understanding  why  people  from  a  different  culture  react  to  events  the  way  they  do, 
detecting  subtle  but  significant  changes  in  attitudes,  forecasting  how  a  situation  will  progress  in  a 
foreign  environment,  and  effectively  planning  how  to  mitigate  these  cultural  reactions  requires 
comprehending  those  interpretations  and  mental  models.  At  the  core  of  understanding 
sociocultural  behavior  is  sensemaking,  which  Klein,  Moon,  and  Hoffman  (2006)  characterize  as  the 
process  of  creating  understanding  in  situations  of  high  complexity  or  uncertainty  in  order  to  make 
decisions.  They  describe  sensemaking  as  "a  motivated,  continuous  effort  to  understand 
connections  (which  can  be  among  people,  places,  and  events)  in  order  to  anticipate  their 
trajectories  and  act  effectively"  (p.  71). 

Through  sociocultural  behavior  sensemaking  we  seek  to  better  understand  events  in  the 
operational  environment  as  well  as  how  to  react.  Understanding  entails  accurate  situation 
awareness  and  option  awareness  about  an  environment.  Endsley  (1995)  defines  three  levels  of 
situation  awareness  which  we  can  apply  to  understanding,  detecting,  and  even  forecasting 
sociocultural  behavior:  (a)  perceiving  the  sociocultural  factors  and  elements  in  an  environment;  (b) 
comprehending  the  impact  of  a  pattern  of  such  factors  on  behavior;  and  (c)  projecting  future 
behavior  based  on  this  comprehension.  Similarly,  Klein,  Drury,  Pfaff,  and  Moore  (2010)  defined 
three  levels  of  option  awareness  to  facilitate  mitigation  planning:  (a)  perceiving  the  relationship 
between  courses  of  action  and  their  impact  on  sociocultural  behavior;  (b)  comprehending  the 
effect  of  sociocultural  factors  in  mediating  that  impact;  and  (c)  projecting  changes  in  outcomes  that 
will  result  from  changes  to  action-options  based  on  that  comprehension.  This  book  uses  these 
definitions  of  sensemaking,  individual  understanding,  situation  awareness,  and  option  awareness 
throughout. 

This  book  describes  the  state-of-the-art  for  sociocultural  sensemaking:  the  most  advanced 
techniques,  technology,  methods,  and  theory  at  this  time.  Such  state-of-the-art  capabilities  are 
found  mostly  in  government,  industry,  and  university  laboratories  -  development  at  the  cutting 
edge,  in  the  pipeline,  but  not  yet  packaged  for  use  in  an  operational  environment. 

The  book  is  divided  into  four  sections  that  address  the  operational  processes  defined  in  Schmorrow 
(2011),  which  he  called  "operational  capability  areas":  gaining  sociocultural  understanding  of  a  new 
area  of  operation,  detecting  sociocultural  factors  and  elements  in  an  environment,  forecasting 
behaviors,  and  mitigating  those  behaviors  in  ways  favorable  to  our  operational  objectives.  Each  of 
these  sections  of  the  book  contains  four  chapters,  which  focus  respectively  on  four  classes  of 
technology:  data  processing,  computational  modeling,  visualization,  and  training. 
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These  chapters  show  how  the  potential  impact  of  these  four  classes  of  technology  differs  across 
the  four  operational  capability  areas.  For  example,  to  aid  understanding ,  visualizations  must  depict 
quantitative  and  qualitative  data  in  a  way  that  provides  situation  awareness  of  a  society's  structure 
and  the  events  that  engage  a  society  through  its  social  networks.  On  the  other  hand,  to  aid 
mitigation ,  visualizations  must  promote  option  awareness  by  presenting  the  landscape  of  plausible 
outcomes  for  each  option  in  a  way  that  enables  the  user  to  easily  compare  those  landscapes  and 
choose  the  most  robust  option.  To  support  detection,  data  processing  must  facilitate  the  collection 
of  information  about  sentiment,  beliefs,  opinion,  human  networks,  and  target  audiences,  and 
collection  must  be  able  to  operate  in  otherwise  denied  or  difficult  to  penetrate  areas.  After 
collection,  data  processing  must  then  enable  the  transformation  of  this  data  into  structures  that, 
for  example,  supply  data  to  the  simulation  models  used  in  any  of  the  operational  capability  areas. 
In  performing  detection ,  warfighters  use  models  to  perform  sentiment  analysis  or  to  identify 
previously  unknown  groups  and  individuals  that  espouse  terrorist  ideologies  and  variants  of 
current  ideologies.  Models  that  support  forecasting  can  project  the  spread  of  an  ideology 
throughout  a  region,  while  those  supporting  mitigation  can  project  how  different  courses  of  action 
may  affect  adversary  decision  making.  Finally,  training  technologies  must  improve  people's 
performance  in  each  of  the  operational  capability  areas,  while  reducing  training  time  and 
resources.  Moreover,  training  must  improve  understanding  of  the  full  range  of  sociocultural 
behavior,  including  interpersonal  skills  that  are  sensitive  to  local  customs,  as  well  as  planning  and 
decision  making  that  take  local  sociocultural  mental  models  into  account. 

Section  One  of  this  book  explores  how  each  of  the  four  classes  of  technologies  contributes  to 
understanding  by  laying  down  a  foundation  of  knowledge  needed  prior  to  initiating  operational 
activities  in  a  given  location.  Dingwall,  Elson,  and  Moaddel's  chapter  on  data  processing  examines 
approaches  to  collecting  behavioral  and  environmental  data  under  difficult  or  even  dangerous 
circumstances.  Brashears  and  Sallach's  chapter  on  computational  modeling  discusses  a  sampling  of 
modeling  techniques  in  terms  of  the  factors  a  tool-user  should  consider  when  identifying  the  most 
appropriate  technique  to  represent  and  convey  sociocultural  understanding.  The  data  visualization 
chapter  by  Ryan  explores  not  only  traditional  means  of  displaying  sociocultural  data,  but  also  new 
techniques  for  visualizing  understanding  in  multi-dimensional  space  and  the  challenges  presented 
by  big  data.  Finally,  in  their  chapter  on  training,  Behymer,  Mateo,  McCloskey,  and  Abbe  describe 
training  tools  that  promote  interpersonal  skills  by  presenting  trainees  with  situations  and 
interactions  with  simulated  people,  as  well  as  tools  that  use  multi-media  tutorials,  real-world 
imagery,  and  virtual  environments  to  help  end-users  develop  cultural  observational  skills. 

The  chapters  in  Section  Two  apply  the  four  technology  classes  to  sociocultural  detection: 
discovering,  distinguishing,  and  locating  operationally  relevant  sociocultural  signatures  through  the 
collection,  processing,  and  analysis  of  sociocultural  behavior  data  (Schmorrow,  2011).  Irvine's 
chapter  on  data  processing  for  detection  focuses  on  transforming  surveys,  social  media,  imagery, 
and  video  into  information  needed  to  identify  noteworthy  sociocultural  patterns.  The  chapter  on 
computational  modeling,  by  Sanfilippo,  Bell,  and  Corley,  describes  using  computational  models  to 
process  data  signatures  and  characterize  the  behavioral  patterns  of  interest  in  the  underlying  data. 
In  their  chapter,  Fricker,  Buttrey,  and  Evans  discuss  how  visualization  can  make  self-evident  the 
sociocultural  phenomena  users  wish  to  discern.  They  highlight  exploratory  data  analysis,  as  well  as 
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simplifications  that  help  focus  attention  on  differences  across  a  data  collection  or  on  changes  in  the 
data  that  occur  over  time.  Finally,  Glazer,  Saner,  Barnes,  and  Pavisic  explore  the  knowledge 
requirements  for  conducting  a  robust  sociocultural  analysis,  and  how  these  requirements 
determine  the  methods  and  designs  needed  to  train  and  educate  intelligence  professionals 
accordingly. 

Section  Three  addresses  the  process  of  tracking  and  forecasting  change  along  multiple  dimensions 
in  sociocultural  entities  and  phenomena  of  interest.  In  his  chapter,  Gabbay  distinguishes  data 
processing  for  forecasting  from  its  use  for  understanding  sociocultural  systems  or  detecting 
sociocultural  signatures.  In  the  context  of  forecasting,  he  identifies  issues  related  to  processing 
data  that  must  be  kept  current  with  actual  events  in  order  to  update  models  that  help  users 
anticipate  changes  along  multiple  dimensions.  In  this  section,  each  contributor  to  the 
computational  modeling  chapter  describes  a  different  type  of  model  for  forecasting.  The  modeling 
techniques  described  include  results  from  mixed-method  aggregation,  forecasting  emotions  from 
social  media,  causal  Bayesian  models,  systems  dynamic  models,  agent-based  models,  hybrid 
models,  "soft"  modeling  methods,  forecasting  of  social  networks,  and  "judgmental"  forecasts.  In 
his  visualization  chapter,  Lustick  then  describes  how  the  necessarily  complex  results  of  such  models 
can  drive  the  standardized,  but  still  dauntingly  complex,  data  visualizations  needed  to  convey 
understanding  to  users.  Finally,  Sieck's  chapter  presents  the  current  state  of  the  science  and 
technology  related  to  training  in  sociocultural  forecasting,  and  identifies  gaps  that  research  must 
bridge  to  develop  capabilities  sufficiently  for  operational  settings. 

The  chapters  in  Section  Four  discuss  methods  for  applying  the  four  classes  of  technology  to 
mitigate  the  influence  of  adverse  sociocultural  behavior  in  the  conduct  of  a  mission.  To  accomplish 
this  mitigation,  commanders  must  develop,  prioritize,  execute,  and  measure  courses  of  action 
(COAs)  grounded  in  the  social  and  behavioral  sciences  (Schmorrow,  2011).  Moreover,  making 
robust  mitigation  decisions  in  today's  complex,  uncertain  environment  demands  more  than 
situation  awareness;  decision  makers  must  achieve  option  awareness  (Pfaff,  Klein,  Drury,  Moon  & 
Entezari,  2012).  Because  decision  making  for  mitigation  relies  so  heavily  on  computational 
modeling,  the  data  processing  chapter  by  Duong  and  Pearman  centers  on  the  representation  of  the 
data  that  are  input,  output,  and  traded  between  models  and  the  interfaces  among  the  models, 
noting  that  interfaces  among  disparate  models  can  be  so  complicated  that  their  design  is  itself  an 
act  of  modeling.  Sliva's  chapter  on  computational  modeling  examines  a  sampling  of  computational 
approaches  that  may  facilitate  COA  development,  analysis,  and  comparison.  The  chapter  describes 
how  computational  modeling  can  relieve  decision  makers  of  some  of  the  information  processing 
burden,  allowing  them  to  synthesize  and  integrate  the  models,  knowledge,  and  insights  from  the 
understand,  detect,  and  forecast  phases  of  analysis  to  provide  better  option  awareness.  Improving 
human  comprehension  of  complex  data  is  also  the  goal  of  the  visualization  techniques  discussed  in 
Yost's  chapter.  Yost  describes  the  research  to  date  in  this  area  and  promising  directions  in  this 
nascent  area  of  using  interactive  visualization  to  help  compare  and  contrast  COAs.  The  training 
chapter  by  Ruvinsky,  Hingst,  Floffman,  and  Kettler  focuses  on  defining  the  skills  (abilities  necessary 
to  perform  the  task)  and  knowledge  (facts,  concepts,  and  principles  needed  to  perform  the  task) 
for  making  effective  mitigation  decisions.  An  initial  analysis  of  the  functional  tasks  involved  in 
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mitigation  stresses  several  key  training  considerations,  such  as  appropriate  objectives  and  ideal 
training  formats  (e.g.,  computer-based  training,  classroom,  or  exercise). 

We  hope  that  the  authors'  surveys  of  state-of-the-art  developments  will  raise  the  operational 
community's  awareness  of  sociocultural  technical  capabilities.  We  hope  they  also  provide  the 
research  and  development  funding  community  with  a  map  of  technology  gaps  that  merit  future 
investments  to  transition  these  capabilities  to  the  field. 
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Section  One:  Understanding 

Gaining  sociocultural  understanding  of  a  new  area  of 
operation 

Jill  Egeth,  The  MITRE  Corporation 


The  sociocultural  behavior  capability  areas  framework  that  guides  the  structure  of  this  book  begins 
its  operational  cycle  with  the  requirement  to  understand  the  sociocultural  environment.  In  order  to 
achieve  understanding,  analysts,  warfighters,  and  decision  makers  at  all  levels  need  capabilities 
that  support  thorough  perception  and  comprehension,  grounded  in  social  and  behavioral  science, 
of  the  sociocultural  features  and  dynamics  in  an  operational  environment  (Schmorrow,  2011). 

As  noted  in  the  introduction,  understanding  involves  both  situation  awareness  and  option 
awareness  of  an  environment.  Endsley  (1995)  would  include  understanding  the  impact  of  a  pattern 
of  sociocultural  factors  on  behavior  in  her  definition  of  situation  awareness,  while  Klein,  Drury, 
Pfaff,  and  Moore  (2010)  would  include  comprehension  of  the  effects  of  sociocultural  factors  on 
action-outcomes  in  a  delineation  of  option  awareness  levels  for  mitigation  planning.  The 
framework  that  guides  this  book  describes  an  organizational  operational  process,  rather  than  the 
cognitive  processing  mechanisms  of  the  brain  -  that  is,  it  delineates  a  standardized  approach  to  a 
process  that  ensures  consistent  behaviors  and  approaches  across  the  members  of  an  organization. 
This  approach  reflects  the  importance  of  laying  down  a  foundation  of  knowledge  and 
understanding  prior  to  initiating  operational  activities  in  a  given  location.  Even  cognitive  process 
models,  such  as  Ulric  Neisser's  perceptual  cycle,  describe  a  psychological  process  in  which 
"anticipatory  schema"  prepare  the  perceiver  to  detect  certain  kinds  of  new  information  that  then 
drives  the  modification  of  a  person's  understanding. 


Data  Processing 

Perceiving  and  understanding  sociocultural  behavior  and  the  sociocultural  environment  is  complex 
and  difficult.  Collecting  data  in  the  operational  domain  presents  challenges:  behavioral  and 
environmental  data  is  frequently  difficult  and  dangerous  to  collect,  access  to  the  populace  may  be 
limited  or  denied,  and  cultural  and  language  differences  act  as  barriers.  The  chapter  on  data 
processing,  by  Elson,  Dingwall,  and  Moaddel  reviews  some  of  the  challenges  and  addresses  two 
developing  approaches  for  achieving  sociocultural  understanding  in  operational  domains.  The  first 
approach  focuses  on  two  research  methodologies  -  content  analysis  and  survey  research  -  that 
support  the  understanding  of  variables  such  as  population  values,  attitudes,  and  ideologies.  The 
second  centers  on  burgeoning  techniques  for  processing  and  understanding  the  output  of  mass 
communication  tools  such  as  Twitter.  The  authors  discuss  both  the  challenges  of  cross-cultural 
research  and  promising  recent  innovations. 
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Computational  Modeling 

Humans  are  social  actors  who  co-create  complicated,  adaptive  systems;  models  of  their 
orientations,  actions,  and  environments  can  benefit  from  advanced  methods  and  mechanisms.  This 
chapter  presents  a  sampling  of  the  models  and  modeling  techniques  currently  in  use  or  under 
development  that  can  represent  social  actors  at  both  the  individual  and  group  levels.  The  chapter 
focuses  on  enhancements  of  various  network,  cognitive,  and  game-theoretical  algorithms  and 
models.  Discussion  of  each  technique  addresses  the  factors  a  tool-user  should  consider  when 
identifying  the  most  appropriate  technique,  including  assumptions  about  actor  knowledge,  data 
requirements,  the  representation  of  human  activity,  the  level  of  analysis  implied  by  the  model,  and 
how  the  model  adapts  to  the  system/environment. 


Visualization 

Data  visualization  exploring  the  interconnections  among  social  and  cultural  forces  that  manifest 
themselves  in  a  variety  of  personal  interactions  is  a  complex  and  daunting  task.  Once  sociocultural 
data  have  been  identified,  collected,  processed,  and  used  as  input  for  computational  models  or  to 
typify  a  snapshot  of  social  interactions,  the  generated  output,  be  it  quantitative  or  qualitative, 
benefits  from  a  visual  display  commensurate  with  the  captured  dynamics  at  play.  Without  good 
visualization  tools  and  techniques,  translating  data  into  situation  awareness  and  into  useful 
recommendations  becomes  problematic.  Ryan's  chapter  describes  methods  for  visualizing 
quantitative  and  qualitative  data,  whether  to  provide  organizational  insights  into  the  structure  of  a 
society  or  to  characterize  events  that  engage  a  society  through  its  social  networks.  In  addition  to 
the  more  traditional  means  of  displaying  sociocultural  data,  the  chapter  examines  new  techniques 
for  visualizing  understanding  in  multi-dimensional  space  and  the  challenges  presented  by 
visualization  using  the  next  wave  of  data  amalgamation  -  big  data.  These  techniques  afford 
analysts  access  to  a  multiplicity  of  media  made  available  through  technological  innovations 
currently  still  in  the  experimental  domain. 


Training 

Achieving  operational  success  depends  not  only  on  the  ability  of  researchers  to  collect  and  process 
data,  model  the  sociocultural  environment,  and  visualize  input  and  output,  but  also  on  the  ability 
of  operational  end-users  to  gain  access  to  the  right  data,  analyze  those  data,  use  the  data  as  input 
into  models,  and  understand  the  models'  output.  As  is  the  case  with  all  new  tools  and  techniques, 
successful  application  and  use  requires  training.  This  chapter,  by  Behymer,  Mateo,  McCloskey,  and 
Abbe,  describes  two  categories  of  training  tools  -  those  that  focus  on  interpersonal  interaction  and 
those  that  address  group-  or  population-level  dynamics.  Training  tools  that  emphasize 
interpersonal  interaction  use  text,  video,  or  virtual  environments  to  simulate  situations  and 
interactions  with  others,  while  the  tools  focusing  on  group  dynamics  use  multi-media  tutorials, 
real-world  imagery,  and  virtual  environments  to  help  end-users  develop  observational  skills  that 
will  help  them  interpret  and  understand  unfamiliar  sociocultural  contexts. 
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1  Sociocultural  approaches  to  understand  human 

interaction:  A  discussion  of  new  theoretical  frameworks, 
issues,  and  modern  communication  technology1 

Sara  Beth  Elson  &  Alison  Dingwall,  The  MITRE  Corporation 
Mansoor  Moaddel2,  University  of  Maryland,  College  Park 


1.  Introduction  to  Sociocultural  Understanding 

The  sons  of  Adam  are  limbs  of  each  other, 

Having  been  created  of  one  essence. 

When  the  calamity  of  time  affects  one  limb 
The  other  limbs  cannot  remain  at  rest. 

If  you  have  no  sympathy  for  the  troubles  of  others, 

You  are  unworthy  to  be  called  by  the  name  of  a  Human. 
-Sa'adi  Shirazi,  13th  century 


No  man  is  an  island  entire  of  itself;  every  man  is  a  piece  of  the  continent,  a  part  of  the  main; 
if  a  clod  be  washed  away  by  the  sea,  Europe  is  the  less,  as  well  as  if  a  promontory  were, 
as  well  as  any  manner  of  thy  friends  or  of  thine  own  were;  any  man's  death  diminishes  me, 
because  I  am  involved  in  mankind.  And  therefore  never  send  to  know  for  whom  the  bell  tolls; 
it  tolls  for  thee. 

-John  Donne,  1624 

That  these  luminaries  from  diverse  cultures  and  times  arrived  at  the  same  conclusion  regarding  the 
psychic  unity  of  mankind  may  lend  credence  to  the  premise  that  all  cultures  strive  for  social 
harmony,  compassion,  and  peace.  This  eloquently  articulated,  idealistic  view  of  humanity, 
however,  cannot  be  easily  reconciled  with  the  persistent  misunderstandings  arising  from  cultural 
diversity  and  the  reality  that  conflicts  not  only  occur  among  cultures  but  that  cultures  themselves 
contain  conflicting  elements.  The  clashes  within  and  between  cultures  run  contrary  to  the  concept 
of  unitary,  caring,  and  compassionate  humankind  that  Sa'adi  and  Donne  celebrated. 


1  Approved  for  Public  Release;  Distribution  Unlimited.  Case  Number  14-2487 
This  work  was  supported  by  Department  of  Defense  Contract  W15P7T-13-C-F601 
Copyright  ©  2014  The  MITRE  Corporation. 

2  Many  thanks  to  Julie  de  Jong  for  her  comments  on  this  chapter. 


10  |  Sociocultural  approaches  to  understand  human  interaction 


While  sociocultural  theories  of  human  interaction  abound,  this  chapter  focuses  on  two  basic 
approaches  for  gaining  sociocultural  understanding.  The  first  approach  uses  a  theoretical 
framework  to  foster  medium-term,  a  period  between  10  and  15  years,  sociocultural  understanding, 
while  the  second  employs  the  new  technologies  of  mass  communication  to  enable  rapid 
understanding  of  how  information,  ideas,  and  propaganda  flow  through  various  outlets  as  well  as 
the  manner  in  which  the  information  flow  shapes  political  actions,  including  extremism. 

The  theoretical  framework  underlying  the  first  approach  illustrates  how  sociopolitical  and  cultural 
issues  serve  as  indicators  of  trends  in  values  and  ideological  change  in  the  medium-term.  In 
describing  this  framework,  we  discuss  two  research  methodologies  that  support  it:  content  analysis 
and  survey  research.  Content  analysis  applies  to  data  collected  in  real  time  from  social  media 
outlets  that  reveals  people's  perceptions  of  issues  and  events.  It  provides  quick  information  for 
understanding  changes  in  the  sentiments,  emotions,  and  attitudes  of  the  subject  population  in  the 
short  run  and  how  such  changes  are  linked  to  different  forms  of  political  action.  With  regard  to 
survey  research,  we  focus  on  the  challenges  of  conducting  cross-cultural  surveys  and  on 
innovations  in  survey  techniques  that  are  not  yet  in  widespread  use. 

Section  three  in  this  chapter,  Technologies  to  Promote  Near-Term  Sociocultural  Understanding, 
focuses  on  innovative  tools  that  can  help  researchers  gain  understanding  of  sociocultural  issues  in 
near-real  time.  Many  of  the  tools  described  are  currently  in  development  or  are  used  only  by  select 
organizations,  but  should  become  more  broadly  available  in  the  near  future.  We  then  discuss  gaps 
in  the  technologies  that  research  should  address  in  the  future.  We  conclude  by  describing  the 
actions  necessary  to  bring  state-of-the-art  technologies  and  methods  into  operational  use. 

We  must  emphasize  the  importance  of  understanding  culture  as  an  overall  framework  for  viewing 
the  world.  The  interaction  between  society  and  an  individual  can  influence  the  development  of 
self-schemas  through  changing  expectations  regarding  self-awareness  (i.e.,  individualist  vs. 
collectivist  societies),  social  judgments  (i.e.,  adhering  to  group  norms),  and  social  behaviors.  But 
just  as  our  own  sociocultural  perspectives  affect  our  own  thoughts,  feelings,  and  behaviors,  it  can 
also  color  how  we  view  other  people  or  groups.  Therefore,  it  is  important  to  recognize  how  social 
context  and  sociocultural  perspectives  may  influence  individual  or  group  behavior.  In  the  section  of 
the  book  covering  detection  of  sociocultural  signatures,  a  chapter  on  training  presents  additional 
discussion  of  culture  as  an  overall  framework. 


2.  Theoretical  Framework  for  Medium-Term  Sociocultural  Understanding 

2.1.  Sociopolitical  and  Cultural  Issues  as  Indicators  of  Trends  in  Values 

"Social  change"  is  an  umbrella  term  used  to  capture  such  observable  aspects  of  social  life  as 
changes  in  (a)  the  principles  of  social  organization  (e.g.,  economics  and  politics,  the  form  of 
government,  or  the  relationship  between  religion  and  politics);  (b)  religious  beliefs  and  institutions 
(e.g.,  the  rise  of  different  religious  movements,  sects,  and  cults);  (c)  values,  rituals,  identity,  and 
life-style  (significant  changes  in  people's  religiosity,  consumption  habits,  and  style  of  dress);  and  (d) 
the  arts  and  literature.  The  actual  process  of  change,  however,  is  complex,  and  its  probable  forms 
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and  content  are  hard  to  detect.  In  some  cases,  change  appears  chaotic,  exhibiting  few  discernible 
patterns;  only  multifaceted  group  conflicts,  sociopolitical  upheavals,  violence,  and  contradictory 
events  are  evident.  In  other  cases,  by  contrast,  change  lies  hidden  beneath  the  veneer  of  political 
stability  and  smooth  economic  transition.  A  sudden  emergence  of  revolutionary  movements 
leading  to  the  breakdown  of  the  existing  political  order  and  changes  in  social  relations  can  interrupt 
a  stable  and  quiet  period  of  economic  development.  In  the  first  instance,  we  know  that  changes  are 
transpiring,  but  we  do  not  know  the  direction  of  these  changes  and  what  cultural  pattern  will 
predominate.  In  the  second,  we  expect  the  emergent  cultural  pattern  to  correspond  to  the 
observable  indicators  of  economic  development  and  political  formation  and  are  surprised  when  a 
strong  countercurrent  surfaces. 

We  suggest  that  certain  indicators  in  public  opinion  and  attitudes,  as  well  as  in  the  expression  of 
intellectual  leaders  and  opinion  makers,  offer  reliable  measures  of  the  extent  of  support  for 
existing  sociopolitical  and  cultural  orders.  These  indicators  may  change  in  finite  ways,  and  the 
direction  of  changes  in  these  indicators  provides  clues  about  the  possible  content  and  form  of 
social  change  in  a  period  encompassing  less  than  a  generation— the  medium  run.  We  also  suggest 
that  historically  significant  issues  function  as  indicators  and  barometers  of  change  and  that  the 
manner  in  which  issues  are  resolved  signifies  the  direction  of  changes  in  some  or  all  aspects  of 
culture.  In  some  cases,  the  resolutions  entail  revolutionary  change,  causing  the  existing  societal 
model  to  be  abandoned  or  modified  in  favor  of  another.  For  example,  in  the  years  preceding  the 
Iranian  Constitutional  Revolution  (1905-11),  the  arbitrary  power  of  the  monarch  became  a  political 
issue  among  intellectual  leaders  and  activists.  Cultural  change  occurred  when  the  revolution 
substituted  a  constitutional  modality  of  politics  for  monarchical  absolutism.  In  the  years  preceding 
the  Iranian  Revolution  of  1979,  on  the  other  hand,  the  pro-Western  monarchy-centered  secular 
modality,  rather  than  the  monarch's  authoritarianism,  was  the  predominant  point  of  contention. 
The  revolution  resolved  it  by  substituting  clerical  absolutism  for  monarchical  dictatorship 
(Moaddel,  1993;  2005). 

In  other  cases,  issue  resolution  takes  the  form  of  a  rather  gradual  shift  toward,  for  example,  gender 
equality  or  gender  hierarchy,  democracy  or  authoritarianism,  liberal  values  or  religious 
government,  and  national/ethnic  or  religious  identity.  These  changes  may  then  shape  the 
probability  distribution  of  the  success  of  different  political  groups,  movements,  or  political  parties. 

The  model  proposed  here  parameterizes  issue  resolution  in  terms  of  the  variable  features  in  the 
dynamic  context  of  ideological  debates  and  religious  disputes  among  both  intellectual  leaders  and 
the  public  at  large,  on  the  one  hand,  and  the  role  of  the  state  as  well  as  politically  powerful  groups 
in  this  context,  on  the  other.  While  culture  is  conceptualized  "as  a  'tool  kit'  of  symbols,  stories, 
rituals,  and  world-views"  (Swidler,  1986,  p.  273),  we  argue  that  it  hardly  constitutes  a  consistent 
set  of  values,  norms,  rituals,  symbols,  memories,  and  institutional  principles.  Rather,  it  contains 
incongruent  elements  that  often  clash.  In  fact,  fundamental  cultural  values  derive  their  meaning 
from  opposition  to  other  cultural  values  that  may  exist  in  the  same  culture.  This  means  that 
multiple  cultural  schemas  exist  simultaneously  in  a  social  environment  and  can  convey 
contradictory  messages  about  values,  goals,  and  appropriate  methods.  Given  these  multiple, 
potentially  conflicting  schemas,  one  cannot  claim  that  any  particular  culture  dictates  certain  types 
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of  decision  making  and  behavior.  Instead,  as  discussed  by  Swidler  (1986;  2001),  cultural  models 
generally  provide  goals  and  strategies  from  which  people  can  choose  in  making  specific  decisions. 
Thus,  individuals  can  follow  one  set  of  cultural  elements  in  a  particular  circumstance  and  another 
set  of  elements  in  a  different  circumstance.  This  is  true,  because  "people,  manifestly,  care  about 
more  than  one  thing— indeed,  are,  simultaneously  and  sincerely  attached  to  values  that 
clash"(Sniderman,  1993,  p.  224). 

The  existence  of  diverse  cultural  schemata  that  shape  different,  if  not  conflicting,  strategies  of 
action  creates  the  potential  for  perpetual  conflicts  in  society.  This  diversity  complicates 
intracultural  and  intercultural  understanding.  Nonetheless,  sociopolitical  stability  is  ensured  when 
a  subset  of  cultural  elements  is  formalized  into  a  dominant  discursive  framework,  becomes  the 
guiding  principles  of  social  interaction,  and  gains  the  appearance  of  permanence  and  taken-for- 
granted  features  of  social  order.  Certain  social  institutions  and  watchdogs  maintain  and  protect  the 
dominant  discourse,  which  is  ultimately  sanctioned  by  state  power. 

2.2.  Cultural  Debates,  Religious  Disputations,  Political  Conflict,  and  Issue 
Resolution 

Cultural  change  may  occur  when  individuals  or  groups  challenge  some  or  all  of  the  elements  of  the 
dominant  discourse,  these  elements  are  turned  into  issues,  intellectual  leaders  offer  resolutions  to 
the  issues,  and  a  significant  section  of  the  population,  members  of  powerful  groups  or  social 
classes,  or  the  ruling  elite  accept  those  resolutions.  Thus,  an  important  aspect  of  understanding 
cultural  change  entails  understanding  the  actual  process  involved  in  resolving  issues. 

The  emergence,  debate  over,  and  resolution  of  issues  constitute  a  non-anonymous  and  observable 
process.  The  process  begins  when  producers  of  sociopolitical  ideas  turn  such  aspects  of  daily  life  as 
people's  social  identity,  the  status  of  the  economy  and  distribution  of  economic  resources,  poverty, 
social  and  cultural  inequality,  arbitrary  rule,  national  security,  the  perceived  status  of  a  nation  or 
community  vis-a-vis  others,  or  people's  spiritual  needs  into  topics  of  debate.  What  requires 
explanation  is  how  intellectual  leaders  offer  resolutions  to  these  issues.  How,  for  example,  do  they 
resolve  issues  related  to  identity  and  economic  difficulties?  Definition  of  identity  may  be  resolved 
in  terms  of  religion,  nationality,  ethnicity,  or  language.  Similarly,  societies  may  address  economic 
difficulties  in  socialist  terms  at  one  extreme,  or,  at  the  other,  by  giving  tax  incentives  to  the  rich  in 
order  to  promote  investment  in  the  country. 

The  actual  resolution  of  issues  may,  however,  derive  directly  neither  from  the  nature  of  these 
issues  nor  from  the  social  positions  of  the  individuals  involved  in  the  cultural  debates.  To  be  sure, 
social  position  constrains  expression.  An  intellectual  leader  from  a  humble  background  may  be 
more  sympathetic  to  a  socialistic  idea  than  to  the  idea  of  competitive  capitalism.  Nonetheless,  that 
same  person  living  under  Soviet  totalitarianism  might  turn  into  an  enthusiast  of  the  free  market 
capitalism.  To  give  another  example,  people  born  in  Egypt  may  have  only  three  possibilities  for 
defining  their  primary  identity:  Egyptian,  Arab,  or  Muslim/Christian.  Nonetheless,  it  is  crucially 
important  to  understand  how  under  certain  historical  circumstances  Egyptians  define  themselves 
primarily  in  national  territorial  terms  (i.e.,  as  Egyptians),  but  under  other  circumstances  in  terms  of 
language  (i.e.,  as  Arabs)  or  religion  (i.e.,  as  Muslims/Christians). 
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2.3.  Ideological  Target  as  a  Key  Factor  in  Issues  Resolution 

We  argue  that  societies  resolve  issues  within  the  dynamic  context  of  intellectual  debates,  religious 
disputes,  and  political  conflict  as  each  side  of  the  debate  structures  the  sociopolitical  expression  of 
the  other  side.  Each  side  constitutes  the  target  of  ideological  production  for  the  other  side.  In  a 
national  context,  where  the  key  actors  are  the  regime  and  its  opponents,  political  activists  define 
their  collective  identity,  specify  cultural  framing,  and  resolve  sociopolitical  issues  in  oppositional 
relation  to  the  ideology  and  identity  of  the  state.  The  latter  may  be  national  or  foreign,  secular  or 
religious.  This  oppositional  context  determines  whether  people  define  their  collective  identity  in 
terms  of  the  nation,  language,  ethnicity,  or  religion.  For  example,  territorial  nationalism  in  Algeria 
and  Egypt  emerged  in  an  oppositional  relation  to  foreign  occupation  (the  French  and  British, 
respectively),  and  anti-clerical  secularism  and  constitutionalism  in  Iran  in  the  late  nineteenth  and 
early  twentieth  centuries  emerged  in  oppositional  relation  to  ulama  religious  obstructionism  and 
monarchical  absolutism  (Moaddel,  2005).  Some  discourses,  such  as  Islamic  modernism, 
fundamentalism,  and  various  forms  of  secularism— liberal  nationalism  or  authoritarian 
nationalism— have  formed  comprehensive  cultural  movements.  Other  discourses,  such  as  Arabism 
and  pan-Arab  nationalism,  have  focused  on  language  as  the  organizing  principle  of  cultural 
movements. 

2.4.  Methodological  Tools  for  Collecting  Cultural  or  Attitudinal  Data 

In  collecting  sociocultural  data,  researchers  can  use  various  methodological  tools,  including 
comparative-historical  investigations,  participant  and  non-participant  observation  in  the  field, 
experiments,  and  discourse  analysis.  Here,  we  focus  on  two  basic  tools  that  have  proven  useful  in 
collecting  cultural  or  attitudinal  data  in  order  to  understand  and  predict  change.  One  centers  on 
analyzing  the  content  of  the  expressions  of  intellectual  leaders,  opinion  makers,  and  bloggers.  An 
analysis  of  these  expressions  through  time  generates  useful  information  for  reconstructing 
intellectual  trends  in  a  society  and  predicting  the  form  of  future  political  and  cultural  movements. 
For  example,  the  discourse  of  pan-Arab  nationalism  that  inspired  the  military  coups  in  Egypt,  Iraq, 
Libya,  and  Syria  in  the  1950s  through  the  1960s  originated  in  the  1920s.  In  the  same  way,  the 
discourse  of  radical  Islamism  most  prevalent  today  was  formulated  in  the  1950s  and  1960s. 

The  other  method  applies  survey  research  to  assess  the  attitudes  and  value  orientations  of  the 
ordinary  public.  Surveys  are  effective  ways  to  generate  scientific  knowledge  about  the  role  of 
culture  in  human  lives  and  the  causes  and  consequences  of  mass  belief  systems.  Longitudinal  and 
panel  survey  data  provide  useful  information  for  understanding  trends  in  values  in  the  population 
of  interest.  The  following  subsections  outline  key  topics  related  to  conducting  successful  surveys. 

2.4.1  Challenges  in  cross-cultural  survey  research 

The  quality  of  survey  data  is  a  function  of  the  extent  to  which  the  researcher  formulates  all  three 
components  of  survey  research  correctly:  the  questionnaire,  sampling,  and  interviews.  These 
criteria  are  particularly  important  in  the  context  of  cross-cultural  survey  research.  We  describe 
below  several  methods  used  to  facilitate  the  conduct  of  cross-cultural  surveys  in  light  of  the 
challenges  that  may  confound  the  findings  and  undermine  the  validity  of  scientific  generalization. 
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Questionnaire  Development.  The  items  in  the  questionnaire  must  be  clearly  linked  to,  and 
constitute  valid  and  reliable  measures  of,  the  theoretical  constructs  that  drive  empirical  research. 
To  this  end,  researchers  must  consider  several  aspects  of  questionnaire  development,  including 
translation  issues,  cultural  sensitivity,  and  item  organization  and  context  within  the  instrument 
itself. 

When  questionnaires  originally  written  in  English  are  used  to  measure  attitudes  and  value 
orientations  across  multiple  non-English  speaking  populations,  researchers  must  ensure  that  the 
translation  does  not  lose  or  distort  meanings.  One  standard  procedure  to  reduce  the  influence  of 
linguistic  differences  entails  (a)  translation  of  the  questionnaire  from  English  to  the  vernacular  by  a 
competent  translator,  (b)  back-translation  of  the  questionnaire  from  the  vernacular  to  English  by 
someone  who  has  not  seen  the  original  English  version,  and  (c)  a  systematic  comparison  between 
the  two  English  versions.  Another  approach,  which  could  either  complement  or  substitute  the 
standard  procedure  to  reduce  cost,  is  to  poll  some  non-English  respondents  regarding  their 
understanding  of  the  questions  during  the  pilot  testing  of  the  questionnaire.  If  responses  are 
consistent  with  the  intention  of  the  question,  then  the  question  can  be  considered  a  valid  measure 
of  the  construct. 

Scholarly  flexibility  and  non-ethnocentrism  are  also  important  considerations  in  questionnaire 
design.  For  example,  in  some  cases  the  version  of  the  questionnaire  in  the  vernacular,  although 
different  from  the  English  version,  more  effectively  taps  into  the  respondents'  sentiment  and 
intentions.  In  such  a  case,  the  researcher  must  modify  the  English  version  to  conform  to  the 
vernacular  version.  For  example,  in  a  recent  cross-cultural  survey  of  populations  from  relatively 
diverse  countries  such  as  Egypt,  Iraq,  Lebanon,  and  Saudi  Arabia,  the  initial  questionnaire  included 
the  question:  "Given  the  opportunity,  how  often  do  you  think  that  men  will  try  to  take  sexual 
advantage  of  women:  always,  sometimes,  rarely,  or  never?"  Both  the  Egyptians  and  Saudis  argued 
that  the  wording  in  Arabic  referring  to  sexuality  would  be  too  sensitive  in  their  home  countries, 
which  led  to  the  revised  wording:  "Given  the  opportunity,  how  often  do  you  think  that  men  will 
harass  women?"  The  revised  question  uses  less  inflammatory  language  while  still  accessing  the 
original  intention  of  the  question.  Additionally,  there  may  be  Western  biases  in  the  types  of  values 
addressed  in  survey  questionnaires.  For  example,  questions  on  religious  tolerance  or  gender 
relations  may  lead  to  a  negative  image  of  Muslim-majority  countries,  as  these  countries  appear  less 
tolerant  of  religions  other  than  their  own  or  more  supportive  of  gender  inequality  and 
hierarchization. 

Studies  have  shown  that  how  questions  are  worded  and  framed,  including  the  choice  and  order  of 
response  categories,  can  have  significant  effects  on  the  responses  (McColl,  et  al.,  2005). 
Furthermore,  while  non-locatable  participants  and  refusals  may  affect  data  quality  (Braver  &  Bay, 
1992),  other  kinds  of  self-selection  biases  that  result  from  the  respondents'  cultural  outlooks  may 
be  more  subtle  and  their  effects  harder  to  assess.  For  example,  the  connection  between  watching 
satellite  TV  and  religious  attitudes  may  become  complicated  if  people  with  conservative  outlooks 
decided  against  installing  satellite  TV  in  their  homes.  Finally,  over-reporting  socially  desirable  and 
under-reporting  undesirable  behaviors/attitudes  is  another  serious  problem  that  may  have 
considerable  impact  on  the  quality  of  the  data  (Cahalan,  1968;  Gordon,  1987;  Flyman,  1944;  Parry 
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&  Crossely,  1950).  In  Muslim-majority  countries,  for  example,  the  social  desirability  bias  emerges 
most  clearly  in  such  questions  as  those  that  solicit  respondents'  attitudes  toward  homosexuality 
and  prostitution.  While  anecdotal  evidence  indicates  the  presence  and  even  tolerance  of  (male) 
homosexuality  and  prostitution  in  these  countries,  at  least  ninety  percent  of  respondents  in 
surveys  of  the  publics  from  these  countries  said  that  such  acts  are  never  justifiable,  which  would 
probably  indicate  social  undesirability  bias.  Furthermore,  asking  people  in  these  countries  whether 
they  had  ever  practiced  homosexuality  would  almost  certainly  anger  the  respondents,  terminating 
the  interview.  Questionnaire  design  must  take  into  account  contextual  effects  and  desirability 
biases  such  as  those  reported  in  the  examples  above  in  order  to  produce  reliable  and  valid  data. 

Sampling.  Samples  of  country  populations  must  be  nationally  representative,  with  all  respondents 
having  a  known  probability  of  selection.  In  many  non-Western  societies,  the  formulation  of  an 
effective  sampling  frame  presents  a  serious  challenge,  particularly  in  the  absence  of  recent  census 
data.  Researchers  must  assess  how  any  changes  in  the  sampling  procedures  as  a  result  of 
unforeseen  factors  might  affect  the  representativeness  of  the  final  sample  used  in  the  study.  These 
may  include  where  the  non-availability  or  inaccessibility  of  detailed  census  data  requires  using 
alternative  spatial  sampling  or  using  the  random-walk  procedure,  or  where  the  outbreak  of  a  war 
or  political  violence  may  make  certain  areas  inaccessible  to  the  interviewers  or  causes  mass 
migration.  When  designing  samples  for  use  in  cross-cultural  survey  research,  researchers  must  also 
consider  comparability  of  the  samples  across  countries  and  across  time  within  countries. 

Interviewers.  Interviewers'  skill  level,  their  understanding  of  the  questions,  and  honesty  will  make 
or  break  the  validity  of  the  entire  process  of  data  collection.  Interviewers  must  be  well  trained  and 
well  paid,  and  the  field  supervisor  must  review  their  work.  In  the  context  of  cross-cultural  survey 
research,  comparable  interviewer  training  is  also  crucial. 

2.5.  Innovation  in  Survey  Methods:  Understanding  the  Causes  of  Popular 
Attitudes 

Although  surveys  have  made  it  possible  to  gain  sociocultural  understanding,  traditional  surveys 
have  been  limited  in  that  they  do  not  allow  analysts  to  understand  the  causes  of  attitudes  that 
respondents  hold.  Only  experimental  methods  can  pinpoint  the  drivers  of  human  behavior  or  the 
reasons  why  people  hold  the  attitudes  they  hold. 

Numerous  texts,  such  as  Shadish,  Cook,  and  Campbell  (2002),  contain  in-depth  discussions  of 
experimental  methods.  In  essence,  an  experiment  is  a  study  in  which  the  researcher  deliberately 
introduces  (or  "causes")  an  intervention  in  order  to  observe  its  effects.  As  an  example,  a  researcher 
might  have  a  theory  regarding  the  aspect  of  a  message  that  makes  it  persuasive  in  a  population  of 
interest-for  the  purposes  of  this  discussion,  some  characteristic  of  the  person  delivering  the 
message,  such  as  his/her  age.  To  test  this  theory,  the  researcher  might  conduct  a  randomized 
experiment.  For  this  experiment,  the  researcher  must  first  recruit  a  sample  of  participants  who, 
ideally,  represent  the  population  of  interest  (e.g.,  intelligence  analysts).  The  researcher  then  might 
use  a  coin  toss  or  random  number  generator  to  assign  each  participant  to  one  of  two  conditions.  In 
the  first  condition,  participants  view  a  message  ascribed  to  an  older  person,  while  in  the  second 
condition  participants  view  exactly  the  same  message,  but  are  told  that  a  younger  person  delivered 
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it.  By  randomly  assigning  participants  to  view  either  the  message  delivered  by  an  older  person  or  a 
younger  one,  the  researcher  can  ensure  that  both  groups  are  reasonably  similar  to  each  other. 
Therefore,  if  one  group  finds  the  message  more  persuasive  than  the  other  group  does,  this 
difference  is  probably  due  to  the  factor  being  tested  (i.e.,  age  of  the  source  delivering  the  message) 
rather  than  to  any  pre-existing  differences  between  the  groups.  In  this  way,  researchers  can 
establish  a  causal  link  between  some  aspect  of  a  message  (e.g.,  age  of  the  source)  and  the 
persuasive  power  of  that  message. 

Although  experiments  have  traditionally  been  conducted  in  academic  labs,  technology  now  enables 
researchers  to  embed  experimental  methods  within  surveys,  thus  combining  the  strengths  of  each 
method.  Namely,  it  is  now  possible  to  explore  causal  relationships  in  human  behavior  via  a  survey, 
enabling  researchers  to  "work  at  scale"  by  examining  a  large  and  representative  sample  of  an 
entire  population.  A  population-based  experiment  uses  survey  sampling  methods  to  obtain  a  set  of 
participants  representing  a  target  population  of  interest  for  a  particular  theory— whether  that 
population  is  a  country,  a  state,  an  ethnic  group,  or  some  other  group.  In  such  experiments,  the 
researcher  randomly  assigns  participants  to  conditions,  and  inserts  interventions  as  in  any  other 
experiment.  New  technologies,  such  as  Computer-Assisted  Telephone  Interviewing  (CATI)  and  the 
internet  itself,  have  made  it  easier  to  randomly  assign  participants  to  conditions  without  asking 
them  to  come  to  a  particular  place  for  initial  screening.  For  this  reason,  researchers  wanting  to 
discover  a  causal  relationship  that  generalizes  beyond  a  narrow  pool  of  participants  are  well 
advised  to  use  population-based  survey  experiments. 

Although  these  experiments  are  administered  to  representative  samples  of  the  target  population 
of  interest,  they  need  not  target  nationally  representative  population  samples,  and  often  have  not 
done  so.  The  population  of  interest  might  be  members  of  a  particular  ethnic  group,  parents  of 
children  under  the  age  of  18,  people  who  watch  television  news,  or  some  other  subset  of  the 
national  population.  This  ability  to  tailor  the  sample  constitutes  the  great  advantage  of  population- 
based  survey  experiments:  researchers  can  test  theories  on  samples  representative  of  the 
populations  to  which  the  theories  are  said  to  apply. 


3.  Technologies  to  Promote  Near-Term  Sociocultural  Understanding 

Up  to  this  point,  we  have  discussed  a  theoretical  framework  and  methodological  approaches  for 
gaining  sociocultural  understanding  that  have  long  traditions  in  the  behavioral  sciences— and  are 
described  in  commensurately  broad  research  literatures.  However,  with  the  advent  of  new  forms 
of  communication  via  the  internet  (e.g.,  social  media),  researchers  have  developed  new 
technologies  for  collecting  the  voluminous  data  generated.  In  particular,  social  media  outlets  such 
as  Twitter,  Facebook,  YouTube,  blogs,  chat  rooms,  and  others  require  a  new  set  of  methods  and 
technological  prototypes  to  conduct  analysis— particularly  when  trying  to  gain  sociocultural 
understanding  in  the  short  term.  We  now  turn  to  these  newer  methods  and  technologies,  in  the 
hope  of  introducing  the  reader  to  technologies  that  are  ready  for  use  but  perhaps  not  yet  widely 
known.  Because  these  technologies  are  at  different  stages  of  readiness  for  use,  we  have  grouped 
them  according  to  whether  they  are  deployed,  whether  they  are  in  prototype  form  but  not 
deployed,  or  whether  they  are  in  a  research  phase. 
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Deployed  Technologies 

3.1.  Topsy 

Topsy  analyzes  tweets  and  web  pages  gathered  from  millions  of  websites,  blogs,  and  social  media 
services.  The  platform  supports  all  languages,  and  combines  a  search  engine  and  analytic  tools  that 
index,  analyze,  and  rank  content  and  trends.  Topsy' s  pipeline  architecture  currently  processes  over 
400  million  posts  every  day  from  multiple  data  sources,  generating  ranked  and  scored  results 
within  a  few  minutes.  Topsy  currently  operates  the  world's  largest  global  index  of  Twitter  data  and 
gives  access  to  conversations  in  near-real  time  as  well  as  to  an  archive  going  back  several  years. 
Customers  that  include  U.S.  Government  agencies,  global  marketing  firms,  and  large  online 
publishers  use  Topsy  to  monitor  activity  by  keyword,  domain,  and  geographic  areas  and  analyze 
content  by  media  type  (links,  photos,  videos),  author,  and  sentiment.  In  addition,  Topsy's  indexing 
and  live-ranking  technologies  process  content  from  social  networks,  helping  users  to  determine  the 
relative  resonance  of  various  topics  and  to  identify  "tipping  points"  in  message  exposure.  Users  can 
also  develop  predictive  trending  metrics  and  conduct  historical  analysis  to  correlate  social  media 
activity  with  real-world  events.  This  analysis  provides  the  ability  to  view  historical  activity  and  to 
compare  activity  levels  for  related  topics  over  long  periods  of  time.  The  platform  supports  any 
language  regardless  of  character  set. 

3.2.  InTTENSITY 

InTTENSITY  has  created  a  suite  of  tools  to  enable  sociocultural  understanding  (inTTENSITY,  2013). 
One  of  them,  the  Social  Media  Command  Center  (SMCC),  combines  a  social  media  harvesting 
service  with  extraction  and  categorization  engines;  the  combination  of  these  provides  an  analytical 
capability  using  a  cloud-based  approach.  The  SMCC  examines  75  million  social  media  sources, 
including  Twitter,  Facebook,  and  a  variety  of  public  blogs  and  pages.  It  specifically  looks  for  spikes 
in  social  media  volume  of  any  sort  compared  to  a  normal  distribution  (as  calculated  and  adjusted 
periodically  by  a  baselining  activity).  For  example,  if  normal  Twitter  frequency  for  a  given  day  and 
hour  is  approximately  1,000  tweets  per  second  and  the  SMCC  detects  a  spike  up  to  3,000  that 
persists  for  a  sustained  period  within  that  timeframe,  the  system  identifies  and  collects  statistics 
about  that  spike.  In  this  way,  researchers  can  use  the  SMCC  to  detect  and  gather  information  on 
any  high-impact  events,  including  mob  violence,  civil  disturbances,  natural  disasters,  or  any  other 
event  where  people  use  their  mobile  devices  to  "check  in." 

SMCC  can  filter  data  by  geography  and  keywords,  enabling  users  to  focus  on  an  event  within  a 
specific  area.  The  system  works  with  unstructured  text  data  and  enables  users  to  manipulate  the 
processed  data  based  on  specific  categories.  Furthermore,  the  classification  technology  recognizes 
and  displays  relationships  between  data  points  and  divides  the  results  into  contextual  groups.  Its 
clustering  functionality  classifies  unstructured  information  into  thematic  groups,  identifies  the 
major  topics  in  documents,  and  recommends  structures  for  classification. 

InTTENSITY  has  also  built  the  InTTENSITY  Analyze  tool,  which  transforms  text  in  social  media, 
emails,  surveys,  and  other  sources  into  insights  regarding  sentiments  and  trends.  Specifically, 
InTTENSITY  allows  users  to  automatically  discover  topics  of  conversation  in  social  media,  emails, 
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surveys,  and  other  sources;  analyze  coverage  in  social  media;  profile  social  media  user  behavior 
and  preferences;  and  measure  the  effectiveness  of  information  campaigns. 

3.3.  Worldwide  Integrated  Crisis  Early  Warning  System  (W-ICEWS) 

W-ICEWS,  developed  by  Lockheed  Martin  with  funding  by  the  Department  of  Defense  (DoD), 
consists  of  technologies  that  monitor,  assess,  and  forecast  the  occurrence  and  evolution  of 
instability  events  throughout  the  world  (Lockheed  Martin,  2013).  W-ICEWS  provides  situation 
awareness  that  is  designed  to  help  produce  planning  products,  crisis  action  planning,  and  post¬ 
execution  analysis.  The  system  comprises  four  components  that  together  perform  near-real  time 
data  extraction  and  analysis  from  news  reports  and  information  generated  by  the  government 
Open  Source  Center  (OSC). 

W-ICEWS  Data  Management  (iDATA)  provisions  W-ICEWS  models  in  near-real  time  with  data  from 
over  6,000  international,  regional,  national,  and  local  news  sources  (Lockheed  Martin,  2013).  The 
software  has  processed  more  than  30  million  news  stories  in  English,  Spanish,  and  Portuguese  from 
the  past  13  years  to  extract  "who,  did-what,  to-who,  when,  where."  iDATA  uses  deep  (BBN  Serif) 
and  shallow  (JabariNLP)  parsing  technologies  to  produce  over  19  million  unique  geolocated  events 
with  an  accuracy  of  greater  than  80  percent.  iDATA  draws  over  300  different  types  of  coded  events 
from  the  CAMEO  (Conflict  and  Mediation  Event  Observations)  taxonomy,  with  each  event  type 
having  an  observer-neutral  intensity  (Goldstein)  score  that  represents  how  hostile  or  how 
cooperative  the  event  is.  The  actors  (country,  sector,  organization,  individual)  involved  in  events 
come  from  dictionaries  of  over  50,000  named  and  time-indexed  entities  as  well  as  over  700  generic 
agents  (e.g.,  police,  government  official,  protestor).  The  iDATA  repository  also  contains  data  from 
30  different  sources  that  contain  primarily  quantitative  data  on  over  175  different  countries. 
Because  many  countries  do  not  publish  reliable  information  (e.g.,  the  gross  domestic  product  of 
Afghanistan),  iDATA  uses  a  type  of  data  imputation  known  as  the  hybrid  copula  method  to  impute 
missing  data  records.  A  copula  is  a  joint  cumulative  distribution  function  that  captures  the 
dependence  among  a  set  of  random  variables  (Hollenbach  et  al.,  2013).  It  is  a  function  that  binds 
together  two  or  more  univariate  marginal  distributions  of  known  form  to  produce  a  new  joint 
distribution  (Trivedi  &  Zimmer,  2005). 

W-ICEWS  Trending,  Recognition,  and  Assessment  of  Current  Events  (iTRACE)  aids  situational 
understanding  through  analysis  and  visualization  of  event  history  trends  and  patterns  generated  by 
iDATA  (Lockheed  Martin,  2013).  It  gives  users  capabilities  to  generate  time  series,  map-based 
views,  trends,  relationships,  matrices,  and  other  visualizations,  with  drill-down  to  underlying 
stories.  iTRACE  provides  an  automated  capability  to  monitor  political  activity  around  the  globe  by 
automatically  converting  news  reports  into  structured  indices  that  reflect  the  character  and 
intensity  of  interactions  among  key  leaders,  organizations,  and  countries— who  is  doing  what  to 
whom,  when,  where,  and  how  around  the  world.  Users  from  various  military,  government,  and 
intelligence  communities  use  iTRACE  to  create  analytical  products.  iTRACE  was  transitioned  into 
U.S.  Strategic  Command's  Integrated  Strategic  Planning  and  Analysis  Network  (ISPAN)  framework  in 
the  spring  of  2012  on  both  the  Secret  Internet  Protocol  Router  Network  (SIPRNet)  and  the  Joint 
Worldwide  Intelligence  Communication  System  (JWICS).  This  capability,  coupled  with  other  W- 
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ICEWS  components,  is  available  on  U.S.  Southern  Command's  unclassified  W-ICEWS  servers  for  test 
and  evaluation. 

W-ICEWS  Forecasting  (iCAST)  provides  the  capability  to  forecast  instability  events  around  the 
globe,  using  a  mixed  methods  modeling  approach  to  forecast  instability  (Lockheed  Martin,  2013) 
iCAST  combines  forecasts  from  heterogeneous  statistical  and  agent-based  models  to  generate  an 
aggregate  forecast  with  accuracy  greater  than  90  percent  (Ibid).  Users  can  drill  down  into  the 
underlying  model  variables  and  data  to  gain  detail  on  the  forecasts  and  can  experiment  with  the 
effects  of  changes  on  the  model  indicators  in  a  "what-if"  scenario. 

Finally,  W-ICEWS  Sentiment  Analysis  (iSENT)  measures  attitudes  and  perception  about  issues, 
people,  and  events  through  sentiment  analysis  from  blogs,  tweets,  and  Facebook  (Lockheed 
Martin,  2013).  It  also  shows  sentiment  propagation  across  the  internet  and  identifies  key  sites  and 
people  in  shaping  opinion  dynamics.  Open  source  digital  content  generated  by  recent  events,  such 
as  the  Arab  Spring  and  London  riots,  creates  a  valuable  source  of  information.  Using  such 
information,  iSENT  can  give  intelligence  analysts  the  tools  to  understand  emerging  regional  trends 
and  sentiment,  predict  threats  from  groups  and  individuals  or  find  the  proverbial  "needle  in  a 
haystack."  Its  analysis  algorithms  are  designed  to  distinguish  between  useful  information  and  noise 
in  a  rapidly  changing  content.  iSENT  developers  have  conducted  experiments  with  operational 
organizations,  transitioned  the  capabilities  for  operational  use  by  intelligence  analysts  and  desk 
officers,  and  integrated  with  other  W-ICEWS  components  and  the  ISPAN  program  of  record. 

3.4.  Marine  Corps  Civil  Information  Management  System  (MARCIMS) 

U.S.  Military  Civil  Affairs  (CA)  teams  currently  lack  the  ability  to  efficiently  share  and  analyze  critical 
information  and  experiences  they  collect  during  engagements  with  the  civilian  environment. 
MARCIMS  aids  effective  Civil-Military  Operations  (CMO)  by  providing  knowledge  of  the  battlespace 
through  mobile  computing  technologies,  semantic  information  and  knowledge  management,  and 
Web-based  geospatial  decision  support  capabilities.  MARCIMS's  technical  approach  begins  with 
data  collection  using  apps  on  iOS  and  Android  smartphones  with  configurable  forms  and  real-time 
submission  of  data  to  a  central  repository  through  local  cellular  or  wireless  networks.  In  addition, 
MARCIMS  displays  field-collected  CIM  data  in  context  with  basemaps  and  overlays  of  relevant 
imagery  and  geo-data.  Users  can  access  geospatial  analysis  and  statistical  reasoning  tools  via 
standard  Web  service  protocols.  Finally,  MARCIMS  enables  users  to  manage  and  analyze 
relationships  between  field  and  reference  datasets  in  a  Semantic  Wiki  interface. 


Prototype  Technologies 
3.5.  TweetTracker 

Arizona  State  University  professor  Huan  Liu  and  his  colleagues  designed  the  TweetTracker  to  assist 
users  involved  in  Humanitarian  Aid  and  Disaster  Recovery  and  other  complex  operations  to  identify 
topics  of  current  concern  to  crowds  (Goolsby  &  Carley,  2013).  Liu  collected  over  200  million 
disaster-  and  crisis-related  tweets.  U.S.  European  Command  has  already  instantiated  the 
TweetTracker  in  its  Social  Media  Dashboard. 
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TweetTracker  helps  users  to  spot  rumor  and  viral  information  by  leveraging  the  capabilities  of  news 
organizations  and  other  sources  that  collect,  sort,  vet,  and  distribute  information.  For  example, 
TweetTracker  can  help  a  user  find  the  top  retweeted  URL,  breaking  news,  influential  people, 
human  sensors  on  the  ground  providing  on-the-scene  reporting,  and  other  items  of  particular 
interest  to  Military  Information  Support  and/to  Operations  (MISO). 

In  addition,  TweetTracker  capitalizes  on  the  fact  that  many  Twitter  users  gather  and  generate  data 
professionally  and  semi-professionally  (i.e.,  newscasters,  bloggers,  etc.).  These  users  collect  vast 
amounts  of  detail  gleaned  from  their  own  informants,  discover  important  events,  and  distribute 
vetted  information  about  events.  TweetTracker  enables  users  to  monitor,  leverage,  and  exploit 
such  broadcast  information,  and  discover  capable  crowd  networks  and  to  analyze  information 
pathways  and  content  for  actionable  intelligence. 

Along  with  the  capabilities  described  above,  TweetTracker  allows  users  to  find  illustrative  tweets,  a 
view  of  the  overall  network,  the  most  retweeted  actors,  the  hashtag  network,  and  the  most  critical 
hashtags  (i.e.,  those  that  co-occur  most  frequently  with  other  hashtags).  At  present,  users  can 
compare  the  numbers  of  tweets  marked  with  different  hashtags.  Furthermore,  users  can  separate 
English  from  non-English  tweets.  Finally,  users  can  display  trends  in  news  articles  and  in  Twitter 
items  on  the  same  topic,  conduct  multi-country  comparisons  of  the  number  of  tweets  per  hour, 
and  track  the  number  of  tweets  mentioning  a  topic  across  time. 

3.6.  Social  Radar 

The  MITRE  Corporation's  Social  Radar  prototype  provides  population-centric  signal-detection 
capabilities  that  can  yield  early  insight  into  diverse  events  such  as  unrest  in  Africa,  natural  disasters 
in  East  Asia,  or  disease  outbreaks.  The  Social  Radar  capabilities  enable  users  to  track  perceptions, 
attitudes,  beliefs,  and  behaviors  of  a  population  as  expressed  via  Twitter,  blogs,  and  news.  Social 
Radar  offers  an  indications  and  warning  capability,  in  which  users  launch  mini-applications  or 
"widgets"  that  may  work  independently  in  a  modular  fashion  or  may  communicate  with  each 
other.  Within  the  Social  Radar  prototype,  researchers  can  utilize  two  applications  to  understand 
the  sentiments  or  emotions  expressed  regarding  a  topic. 

The  Sentimedir  prototype  (Day,  Boiney,  Ubaldino,  &  Brown,  2012)  supports  detection  and  tracking 
of  trends  in  sentiment  as  expressed  in  news  articles  and  comparable  texts.  Users  can  quantify  and 
visualize  the  direction  and  rate  of  change  in  sentiment,  while  controlling  for  various  features  of  the 
text  such  as  topic,  source,  geography,  persons  named,  and  more.  Users  can  design  highly  tailored 
queries  and  save  them  for  efficient  reuse,  updating,  and  collaborative  sharing.  They  can  also  drill 
down  to  individual  sentences  or  documents,  to  view  annotations  of  sentiment  in  context.  The 
technical  core  of  Sentimedir  is  a  set  of  statistical  sequence  models  that  use  semantic  and  lexical 
features  of  individual  sentences  to  infer  three  elements  of  sentiment  expression:  (a)  opinion 
holder— the  person,  organization,  or  metonymous  mention  of  a  geopolitical  entity,  (b)  sentiment 
expression(s)  —specific  terms  or  phrases,  in  context,  that  convey  the  negative  or  positive  polarity 
of  an  opinion  holder's  sentiment,  and  (c)  sentiment  target—  the  concept  or  entity  on  which  the 
opinion  holder's  expression  of  sentiment  is  focused. 
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The  MoodMiner  tool  suite  comprises  a  set  of  behavioral  scientific  analytic  methods  and  an 
automated  prototype  for  understanding  trends  in  emotional  expressions.  At  present,  MoodMiner 
has  been  applied  only  to  Twitter,  but  will  be  extended  to  other  social  media.  This  tool  enables  users 
to  query  tweets  and  plot  trends  in  emotion  levels  in  real  time— both  generalized  emotions  and 
expressions  regarding  specific  targets.  The  emotion  categories  used  within  the  prototype  are 
drawn  from  social  psychological  research  that  examined  linkages  between  the  words  people  use  in 
written  texts  and  the  psychological  states  they  experience.  Specifically,  the  research  used  to 
develop  the  Linguistic  Inquiry  and  Word  Count  (UWC)  framework  within  social  psychology  also 
informed  the  development  of  the  prototype. 

In  addition  to  monitoring  the  emotions  and  sentiments  expressed  toward  a  topic  over  time,  users 
may  wish  to  understand— at  a  glance— the  most  relevant  topics  being  discussed  in  a  set  of  text 
data  or  to  pinpoint  the  demographic  characteristics  of  those  posting  tweets.  Alternately,  users  may 
wish  to  uncover  deception  in  the  tweets.  Each  of  these  analyses  can  be  conducted  within  Social 
Radar. 

Users  can  draw  on  Comment  Filter  (CoFi)  to  perform  exploratory  analysis  of  text  data  sets  in  any 
language  by  using  NLP  techniques  to  group  similar  comments  and  to  prioritize  messages  such  that 
the  most  relevant  items  can  be  identified  quickly  (Doran,  Zarrella,  &  Flenderson,  2012).  CoFi 
automatically  discovers  topics,  grouping  similar  comments,  sorting  comments  by  relevance,  and 
providing  drill-down  and  timeline-based  visualization.  It  works  with  other  analytics  that  detect 
elevated  levels  of  information  sharing  behavior,  such  as  retweets,  and  can  partition  social  media 
comments  by  language  and  country  of  origin  to  provide  a  more  targeted  perspective  to  the  analyst 
interested  in  drilling  down  to  search  messages  for  actionable  content. 

Users  who  wish  to  understand  the  demographic  characteristics  of  those  posting  to  Twitter  can 
draw  on  Author  DNA  to  characterize  unknown  authors  along  several  attributes,  including  gender, 
age,  and  location.  Burger,  Flenderson,  Kim,  &  Zarrella  (2011)  cast  the  problem  as  one  of  text 
classification,  viewing  each  author  as  a  set  of  identifying  features  derived  from  the  content  and 
metadata  associated  with  all  of  the  author's  postings.  These  features  include  the  sets  of  words  and 
characters  used  in  the  tweet  text  itself,  as  well  as  the  author's  self-description,  screen  name,  time 
zone,  posting  times,  text  length,  emoticons,  capitalization  and  punctuation  density,  and  numerous 
other  attributes  associated  with  a  user's  social  activity.  Burger,  Flenderson,  Kim,  &  Zarrella,  (2011) 
built  a  statistical  profile  over  these  units,  and  compared  it  to  known  samples  by  measuring  the 
similarity  of  these  distributions.  This  approach  is  agnostic  about  language  or  writing  system,  making 
it  applicable  to  many  organizations'  problem  spaces. 

Author  DNA  has  so  far  focused  on  machine  learning  algorithms  that  can  be  trained  quickly  on  very 
large  amounts  of  training  data.  Flenderson,  Zarrella,  Pfeifer,  and  Burger  (2013)  have  labeled  more 
than  20  billion  tweets  with  a  country  of  origin,  and  have  performed  a  series  of  validation 
experiments  using  a  sample  of  Twitter  data  built  over  the  course  of  four  years.  They  used  blog 
profile  metadata  and  tweet  geo-tags  to  measure  the  correctness  of  their  predictions,  with  results 
suggesting  that  the  algorithms  are  roughly  92%  accurate  at  associating  a  Twitterer  with  a  country. 
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As  a  practical  matter,  Twitterers  who  produce  more  tweets  are  more  likely  to  be  tagged  more 
accurately,  so  the  approach  is  well  suited  for  discovering  highly  active  Twitterers  from  any  region  of 
the  globe. 

Users  who  wish  to  detect  deception  or  covert  activity  in  social  media  can  turn  to  the  Pinocchio 
widget,  which  searches  for  covert  information  campaigns  operated  by  hidden  networks  of  social 
media  accounts.  "Puppet"  accounts  such  as  these  are  often  used  to  conduct  fake  grass-roots 
campaigns  to  distort  the  online  conversation  or  magnify  perceived  influence.  Pinocchio  searches 
for  a  constantly-updated  list  of  rare  watermarks  that  are  potentially  unique  to  the  style  or 
substance  of  an  individual  user's  messages,  such  as  unusual  phrases,  hashtags,  or  URLs.  When  the 
widget  discovers  multiple  accounts  that  repeatedly  share  the  same  rare  watermarks,  it  flags  those 
users  as  potentially  suspicious.  When  those  flags  start  appearing  between  all  members  in  a  much 
larger  set  of  accounts,  at  statistically  unusual  rates,  Pinocchio  builds  up  a  high-level  picture  of  who 
is  colluding  to  deceive  honest  users  of  social  media. 

Research  Phase  Technologies 

3.7.  Program  on  Social  Media  in  Strategic  Communication  (SMISC) 

According  to  the  website  of  the  Defense  Advanced  Research  Projects  Agency  (DARPA,  2013), 
DARPA  launched  the  Social  Media  in  Strategic  Communication  (SMISC)  program  to  develop  tools 
that  would  help  human  operators  to  counter  misinformation  or  deception  campaigns.  The  program 
focuses  its  research  on  linguistic  cues,  patterns  of  information  flow,  and  detection  of  sentiment  or 
opinion  in  information  generated  and  spread  through  social  media.  Researchers  will  also  attempt 
to  track  ideas  and  concepts  to  analyze  patterns  and  cultural  narratives.  If  successful,  they  should  be 
able  to  model  emergent  communities  and  analyze  narratives  and  their  participants,  as  well  as 
characterize  generation  of  automated  content,  such  as  by  bots,  in  social  media  and  crowd  sourcing. 

SMISC  researchers  are  creating  an  environment  where  large  amounts  of  data  are  collected,  with 
experiments  performed  in  support  of  development  and  testing.  One  example  of  such  an 
environment  might  be  a  closed  social  media  network  of  2,000  to  5,000  people  who  agree  to 
conduct  social  media-based  activities  in  this  network  and  to  participate  in  required  data  collection 
and  experiments.  This  network  might  be  formed  within  a  single  organization,  or  span  several. 
Another  example  might  be  a  role-player  game  in  which  use  of  social  media  is  central  and  players 
have  again  agreed  to  participate  in  data  collection  and  experiments. 


4.  Gaps  in  Science  and  Technology 

Although  research  has  made  great  strides  in  developing  technologies  for  ingesting  and  analyzing 
social  media  data,  certain  gaps  remain.  One  technological  gap  limits  the  ability  to  differentiate 
between  people's  "public"  versus  "private"  selves  from  social  media  data.  In  addition,  given  that 
several  of  the  technologies  discussed  in  this  chapter  include  programs  for  tracking  linguistic 
patterns  in  text  data,  we  note  two  gaps  in  the  area  of  linguistics  and  culture.  A  fourth  gap  concerns 
the  use  of  virtual  reality  games  (or  lack  thereof)  in  research  designed  to  gain  sociocultural 
understanding.  Finally,  a  fifth  gap  lies  in  the  scientific  tradecraft  available  for  utilizing  the  diverse 
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methods  of  data  collection  available  (e.g.,  survey  research,  social  media  analysis,  and  others). 
Specifically,  a  question  arises  as  to  whether  insight  gained  from  social  media  is  analogous  to,  or 
comparable  with,  insight  gained  from  surveys,  content  analysis,  focus  groups,  Mechanical  Turk 
studies,  or  other  forms  of  data  collection.  This  question  is  central  to  determining  whether  a  user 
can  obtain  "ground  truth"  when  utilizing  any  one  of  these  forms  of  data  collection  in  isolation.  We 
will  address  each  of  these  gaps  in  turn. 

4.1.  Detecting  Public  versus  Private  Selves 

As  the  internet  becomes  ever  more  popular  worldwide,  people  increasingly  encounter  "alternate" 
social  environments  and,  therefore,  opportunities  to  experience  different  self-schemas.  Although 
people  seek  to  maintain  a  consistent  view  of  themselves,  the  social  environment  can  elicit 
alternate  behaviors  and  expressions  depending  on  whether  it  triggers  their  "public  selves"  or 
"private  selves."  An  individual's  private  self  engages  in  thoughts,  feelings,  and  ideas  that  form 
his/her  self-schema.  The  public  self,  however,  is  the  identity  revealed  to  others,  and  can  be  context 
specific.  Oftentimes,  the  person  attempts  to  conceal  the  private  self,  while  the  public  self  includes 
the  actions  that  others  can  see  and  acknowledge.  Both  personas  can  affect  behavior.  For  example, 
people  may  follow  social  norms  at  work  or  in  certain  social  contexts;  if  they  do  not,  they  risk  being 
ostracized  by  their  group.  In  addition  to  the  physical  problems  that  result  from  being  rejected  by  a 
group,  ostracism  can  lead  to  adverse  psychological  effects,  including  on  one's  overall  sense  of 
social  well-being  (Williams,  2001). 

Virtual  environments,  however,  make  it  difficult  to  decipher  whether  an  individual  is  displaying 
her/his  public  self  or  private  self.  Thus,  anyone  wanting  to  tap  underlying  beliefs  would  need 
technology  that  can  distinguish  when  people  are  displaying  public  selves  versus  private  selves.  Such 
technology  does  not  exist  at  present. 

4.2.  Psycholinguistics  within  Foreign  Languages 

Several  of  the  automated  technologies  discussed  in  this  chapter  track  linguistic  patterns  in  text 
data— whether  these  patterns  consist  of  the  frequencies  of  words  or  of  phrase  structures  (such  as 
subject-verb-object  "triples").  Although  considerable  basic  research  in  the  English  language  has 
linked  the  usage  of  specific  words  with  the  emotional/cognitive  states  that  people  are  experiencing 
(Tausczik  &  Pennebaker,  2010),  very  little  such  research  has  focused  on  non-Western  languages 
and  cultures.  Thus,  it  is  not  always  straightforward  to  interpret  the  linguistic  patterns  spotted  in 
languages  other  than  English  or  what  these  patterns  signify  regarding  people's  psychological  states. 
Considerable  cross-cultural  research  will  be  needed  to  fill  this  gap  in  scientific  understanding. 

4.3.  Tracking  Linguistic  Patterns  in  Foreign  Languages 

Another  set  of  challenges  arises  because  the  "languages  of  the  internet"  are  often  combinations  of 
different  languages  thrown  together.  For  example,  Twitter  users  in  the  Arab  world  often  combine 
an  Arabic  dialect  with  English  (to  form  what  is  known  as  "Arabeezy")  and  even  throw  in  Twitter 
slang  and  emoticons.  The  combination  of  such  diverse  languages  as  Arabic  and  English  alone  poses 
a  formidable  gap  that  yet-to-be-invented  technologies  may  someday  address. 
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As  languages,  in  and  of  themselves,  have  evolved  through  the  centuries,  the  "internet  languages" 
also  evolve— quite  possibly  at  a  faster  rate.  Therefore,  any  technology  used  to  process  language 
must  account  for  the  constantly  shifting  linguistic  patterns  of  the  internet.  Whether  this  requires  a 
"human-in-the-loop"  to  monitor  new  language  patterns  or  whether  research  will  produce  an 
automated  way  of  detecting  new  patterns,  anyone  wishing  to  analyze  social  media  communication 
in  the  long  term  will  have  to  grapple  with  "constant  change"  on  the  internet. 

4.4.  Virtual  Reality  Games  for  Gaining  Sociocultural  Understanding 

Virtual  reality  simulations  are  available  for  training  purposes,  e.g.,  to  help  troops  gain  sociocultural 
understanding  before  deploying  overseas,  yet  few  virtual  reality  games  exist  for  gaining 
sociocultural  understanding.  Such  games  could  be  designed  and  implemented  in  the  context  of 
internet  experiments  (e.g.,  using  the  experimental  methodology  described  above).  In  this  kind  of 
experiment,  researchers  could  examine  the  decisions  that  participants  make  during  a  game  and  the 
thoughts  they  might  communicate  for  clues  as  to  their  sociocultural  perspectives.  Such  games  have 
the  advantage  of  being  dynamic  and  engaging,  as  well  as  representing  a  consistent  and  systematic 
mechanism  for  data  collection,  since  they  do  not  depend  on  a  human  telephone  interviewer  to 
speak  exactly  the  same  way  in  every  interview.  For  a  full  discussion  of  the  training  tools  available 
for  gaining  sociocultural  understanding,  see  the  Training  chapter  in  the  Understand  section  of  this 
book. 

4.5.  Conducting  Multi-Method  Analysis 

More  than  ever  before,  operational  end-users  have  a  choice  of  data  collection  methods  and 
analytical  processes  for  gaining  sociocultural  understanding.  These  methods  include  those 
described  in  this  chapter  as  well  as  focus  groups,  in-depth  interviews,  on-the-ground  observations, 
and  many  others.  At  present,  however,  few  studies  have  attempted  to  compare  results  across 
different  methods— especially  when  it  comes  to  examining  non-Western  cultures.  A  few  studies 
have  compared  social  media  results  with  surveys  when  it  comes  to  American  domestic  issues 
(Mitchell  &  Hitlin,  2013;  O'Connor,  Balasubramanyan,  Routledge,  &  Smith,  2010). 

This  indicates  a  need  for  systematic  research  to  understand  the  factors  that  may  cause  different 
methodologies  to  converge  on  the  same  findings  or  to  diverge— especially  with  regard  to  foreign 
populations.  For  example,  if  a  social  media  analysis  and  a  survey  project  within  one  culture  produce 
identical  results,  this  "multi-method  convergence"  suggests  that  the  results  are  true.  However,  if 
the  findings  diverge,  where  does  "ground  truth"  lie?  At  present,  little  conventional  wisdom  points 
the  user  toward  a  process  for  making  sense  of  diverging  results.  In  one  culture,  people  may  be 
afraid  to  speak  the  truth  in  a  telephone  survey,  because  they  fear  being  wiretapped,  but  will  readily 
state  their  opinions  on  the  internet.  However,  in  cultures  that  are  more  open,  different  methods 
may  yield  more  convergent  findings.  In  yet  another  scenario,  surveys  might  tend  to  reach  one 
demographic  group,  while  social  media  analyses  may  cover  a  different  demographic.  Again,  this 
pattern  of  demographic  coverage  may  hold  true  in  some  cultures  but  not  in  others. 

Ultimately,  research  comparing  results  across  different  data  formats  can  clarify  how  to  interpret 
the  results  of  one  type  of  study.  It  can  also  suggest  whether  or  not  it  would  be  adequate  to  conduct 
only  one  or  another  type  of  study  in  a  given  culture,  or  whether  it  is  more  effective  to  conduct 
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multiple  types  of  studies.  Such  research  can  ultimately  contribute  toward  an  analytic  tradecraft  and 
best  practices  for  interpreting  the  results  of  different  types  of  studies. 


5.  Transition  to  Operational  End-Users 

Although  numerous  methods  and  tools  now  exist  for  analyzing  social  media  data,  developers  must 
meet  a  series  of  challenges  to  transition  these  tools  effectively  to  operational  environments  (Allen, 
unpublished  manuscript).  For  example,  a  new  technology  may  have  unforeseen  second-order 
impacts  on  business  processes  and  capabilities.  To  understand  the  effects  on  an  analytic  business 
process,  potential  users  must  outline  the  business  process  functions  at  the  most  basic  level  and 
then  postulate  specific  ramifications  ahead  of  time. 

As  one  example,  a  new  technology  provided  to  intelligence  analysts  might  improve  the  efficiency  of 
gathering  data  from  intelligence  sources.  This,  in  turn,  would  simplify  and  speed  up  the  overall 
super-task  of  gathering  intelligence  from  multiple  sources,  permitting  analysts  to  go  beyond 
"Source  A"  alone  to  other  sources.  However,  collecting  intelligence  from  any  source  comprises  a 
set  of  smaller  implicit  tasks,  such  as  "understand  the  data  type,"  "understand  the  context  in  which 
the  data  was  produced,"  "understand  the  technical  limitations  of  the  collector,"  and  "judge  the 
relevance  of  the  data  source  to  the  question."  If  the  analyst  was  already  familiar  with  Source  A, 
each  of  these  sub-tasks  was  encapsulated  in  the  super-task.  By  collapsing  several  iterations  of  the 
super-task  (i.e.,  "Gather  from  Source  B,  ...C,  ..."  etc.),  the  new  technology  exposes  the  finer-grained 
tasks,  because  the  analyst  is  less  familiar  with  the  new  sources.  As  a  result,  while  the  new 
technology  may  consolidate  the  data-gathering  tasks,  it  extends  the  processing  for  validity  and 
relevance  enormously— an  unintended  consequence  (Allen,  2007). 

Another  set  of  challenges  to  effective  transition  of  technologies  lies  in  the  skill  sets  that  analysts 
need  in  order  to  utilize  new  technologies  effectively.  Potential  users  should  carefully  consider 
whether  the  technology  requires  advanced  disciplinary  or  specialized  training  (such  as  a  Master's 
or  PhD  in  a  particular  discipline)  in  order  to  understand  or  use  it.  If  the  technology  makes  use  of 
behavioral  principles  or  mathematical  processes,  for  example,  agencies  may  need  either  to  ensure 
that  someone  trained  in  the  relevant  disciplines  is  on  staff  to  help  analysts  navigate  the  technology, 
or  to  provide  elaborate  training,  manuals,  and  ongoing  support  as  analysts  learn  the  underlying 
principles  for  using  the  technology  appropriately. 

As  one  example,  analysts  must  understand  the  nuances  and  limitations  of  particular  data  types  in 
order  to  synthesize  multiple  data  sources  effectively,  especially  because  most  analysts  "grow  up" 
within  a  particular  intelligence  discipline  (Allen,  2007).  Developing  an  analytic  strategy  across 
different  types  of  data  without  having  familiarity  with  all  of  them,  as  occurs  when  huge  repositories 
become  available,  adds  substantially  to  an  analyst's  tasks.  Furthermore,  as  described  above, 
altering  business  processes  to  incorporate  such  "new"  tasks  has  a  significant  impact  upon  timing, 
planning,  and  production  processes. 

Furthermore,  analysts  must  understand  how  to  develop  query  strategies  in  order  to  pull  the  data 
sets  they  need  from  the  voluminous  amounts  of  data  available  (Allen,  2007).  To  date,  the  typical 
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analyst  is  not  well  versed  in  more  than  Boolean  or  frequency-based  querying.  Development  of 
query  strategies  appropriate  to  the  amount  and  type  of  data,  the  particular  problem  set,  and  the 
technical  requirements  of  the  storage  solution  requires  better  understanding  and  manipulation  of 
semantic,  statistical,  and  conceptual  clustering  strategies.  To  this  point,  tool  developers  have  made 
significant  strides  on  developing  novel  exploitation  algorithms,  but  bridging  the  gap  to  the  user  has 
been  difficult. 

The  final  challenge  relates  to  analysts'  overall  comfort  level  with  using  technology  in  general,  and  in 
particular,  with  going  beyond  Microsoft  Office.  Analysts  might  require  careful  training  in  the  use  of 
any  new  technology,  or  may  need  to  rely  on  excellent  "cultural  brokers"  to  facilitate  the  use  of  that 
technology.  Ultimately,  the  adoption  of  new  technologies  requires  adjustments  to  the  new  tools, 
techniques,  and  processes.  Challenges  concerning  the  skill  levels  of  analysts  may  be  especially 
difficult  to  address,  because  they  involve  changing  people  as  opposed  to  systems  or  processes. 

Separate  barriers  to  effective  technology  transition  lie  in  the  dissemination  processes  currently 
used  by  government  agencies.  For  example,  data-sharing  agreements  made  at  one  agency  may 
incorporate  some  limitations  regarding  the  types  of  users  allowed  to  receive  the  data.  This 
situation  creates  problems  for  analysts  working  in  one  unit  but  supporting  a  Joint  or  Coalition 
activity.  Furthermore,  some  regulations  stipulate  that  data  can  only  be  retained  for  a  specified 
length  of  time,  causing  problems  when  performing  tasks  that  require  historical  data.  Similar 
challenges  arise  when  data  sets  may  be  available  to  answer  a  question  but  are  not  available  for 
review  by  another  organization.  For  example,  an  analyst  within  one  unit  might  solve  an  analytic 
problem,  but  cannot  release  the  relevant  detail  to  the  supported  unit  because  of  the  nature  of  the 
information  used. 


6.  Summary 

In  this  chapter,  we  provided  an  overview  of  theoretical  frameworks  and  technologies  that 
researchers  can  use  to  gain  sociocultural  understanding.  Although  these  frameworks  and 
technologies  are  rigorous,  they  are  not  yet  widely  used  in  operational  contexts.  We  outlined 
technologies  on  the  horizon  for  gaining  sociocultural  understanding  in  the  medium  term  and  for 
conducting  rapid  analysis  of  large  volumes  of  data  for  near  real-time  understanding.  This  latter 
mission  brings  with  it  the  challenge  of  converting  massive  amounts  of  data  into  tractable  units  for 
analysis.  Although  the  technologies  available  do  an  impressive  job  of  exactly  that,  we  highlighted 
gaps  in  these  technologies  and  in  the  science  of  sociocultural  behavior  sensemaking  underlying 
them.  We  also  highlighted  some  possible  shortcomings  that  may  limit  attempts  to  transition 
technologies  into  operational  contexts.  We  hope  that  the  reader  will  come  away  with  a  greater 
understanding  of  the  avenues  available  for  sociocultural  analysis  as  well  as  a  realistic 
understanding  of  the  challenges  involved  in  pursuing  these  avenues. 
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1.  Introduction 

Accurate  knowledge  of  human  behavior  is  necessary  to  deal  with  many  strategic  and  tactical  issues, 
but  attaining  such  knowledge  is  difficult.  Individuals  are  complex,  self-reflective,  and  adaptive. 
Thus,  a  strategy  that  succeeds  at  one  time  may  fail  at  another.  Moreover,  group  behavior  is  not 
simply  individual  behavior  scaled  up;  lessons  learned  at  the  individual  level  may  not  apply  to  a 
group  and  vice  versa.  In  short,  humans  act  and  interact  in  complicated  systems,  and  complex 
models  are  often  necessary  to  describe  them. 

This  chapter  surveys  some  of  the  techniques  that  comprise  the  current  state  of  the  art  in  modeling 
human  behavior  at  both  the  individual  and  group  levels.  The  discussion  is  not  exhaustive,  as  much 
work  is  being  done  in  this  area,  but  it  provides  a  brief  overview  of  some  of  the  most  promising 
methods  currently  in  development.  Specifically,  we  focus  on  various  types  of  network-based 
models,  cognitive  models,  and  game-theoretical  approaches.  Network  analysis  constitutes  an 
important  and  growing  area  of  study  and  is  vital  in  helping  U.S.  forces  to  confront  decentralized 
opposition  groups.  Cognitive  models  have  become  essential  due  to  the  rise  of  self-starter  terror 
cells,  which  plan  and  execute  attacks  with  minimal  or  no  coordination  or  resources  from  a  central 
organization.  Finally,  game-theoretic  approaches  apply  broadly  to  understanding  equilibrium 
conditions  across  a  wide  range  of  levels  of  analysis,  while  promulgame  theory  concentrates  on  how 
game  types  interact  to  produce  outcomes.  To  conserve  space  we  have  omitted  other  useful  topics, 
such  as  agent-based  simulation  models,  system  dynamic  models,  and  uncertainty  analysis;  these 
are  covered  elsewhere  in  this  volume. 

Computational  models  of  human  behavior  differ  in  (a)  the  kinds  of  assumptions  made  about  actors' 
knowledge;  (b)  the  type  of  data  required;  (c)  the  representation  of  human  activity;  (d)  the  level  of 
analysis;  and  (e)  how  adaptation  is  treated,  if  at  all.  We  need  a  firm  grasp  of  each  of  these  factors 
to  select  an  appropriate  model  and  use  it  correctly. 

Assumptions.  All  models  impose  some  type  of  assumptions  on  the  data,  but  those  assumptions  can 
differ  markedly.  For  example,  some  models  assume  that  individuals  have  perfect  knowledge  about 
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other  group  members,  while  other  models  assume  limited  knowledge.  Empirical  reality  frequently 
violates  some  of  the  assumptions  underlying  a  model,  so  users  must  assess  whether  the  violations 
are  serious  enough  to  make  the  model  unreliable. 

Data  Requirements.  Models  vary  in  the  amount,  and  type,  of  data  that  they  require  as  well  as  in 
their  potential  outputs.  Some  models  require  only  very  high-level  demographic  information,  but 
produce  high-level  abstractions.  Other  models  require  detailed  information  on  the  characteristics 
and  relationships  of  each  modeled  actor,  but  produce  reliable  estimates  of  individual  attitudes  and 
behaviors  (e.g.,  technology  adoption).  Thus,  both  the  outputs  desired  and  the  type  and  amount  of 
data  available  drive  model  selection.  However,  there  are  no  free  lunches:  the  amount  of  data  used 
to  feed  a  model  correlates  at  least  roughly  with  the  detail  and  accuracy  of  the  model's  output.  This 
does  not  mean  that  accurate  data  always  yields  an  accurate  model,  but  only  that  it  is  difficult  or 
impossible  for  a  model  to  achieve  a  higher  level  of  precision  than  is  inherent  in  its  inputs.  Similarly, 
the  data  must  accurately  represent  the  process  that  analysts  hope  to  explain.  Biased  or  incomplete 
data  will  produce  biased  or  incomplete  results  even  when  analyzed  using  the  best  available  model. 

Representation  of  Human  Activity  Change.  Human  activity  has  structure,  but  models  differ  in  how 
they  represent  this  structure.  Some  models  assume  that  human  relationships  are  relatively  fixed, 
while  others  allow  relationships  to  be  born,  mature,  and  finally  die.  The  appropriate  representation 
depends  on  the  application;  fixed  relationships  may  be  reasonable  when  modeling  short  spans  of 
time,  but  not  long  spans  of  time. 

Levels  of  Analysis.  Models  can  be  applied  to  levels  of  analysis  that  range  from  individuals,  to  small 
groups,  to  organizations,  nations,  and  even  alliances.  Analysts  should  select  models  in  part  on  the 
basis  of  the  fit  between  the  levels  of  analysis  envisioned  by  the  model  and  the  particular  entities 
studied.  In  many  cases  the  same  model  may  be  appropriate  for  multiple  levels  of  analysis,  but  the 
implications  of  model  output  can  change  in  both  obvious  and  subtle  ways  across  these  levels. 

Adaptation.  Humans  are  self-reflective  entities,  capable  of  understanding  their  surroundings  and 
reacting  to  them.  In  other  words,  individuals  or  groups  may  recognize  an  intervention  as  an 
intervention  and  adjust  their  behavior  or  thinking  in  response.  Computational  models  can 
represent  these  dynamics,  but  such  modeling  must  be  an  ongoing  process.  Either  a  model  of  a 
social  system  should  have  reflexive  capabilities,  or  analysts  must  update  and  revise  it  to  take 
account  of  the  changing  empirical  circumstances. 

In  the  remainder  of  this  chapter  we  explore  several  families  of  models  useful  for  understanding 
social  behavior,  including  Blau  space  ecological  models,  social  network  models,  social  influence 
models,  cognitive  models,  game  theory,  and  promulgame  models.  We  begin  with  Blau  space 
ecological  models,  which  allow  analysts  to  study  network  processes  using  demographic  and 
membership  data.  These  data  are  relatively  easily  obtained  and  thus  Blau  space  approaches  are 
useful  for  producing  high-level  abstractions.  We  then  consider  social  network  and  social  influence 
models,  both  of  which  allow  for  relatively  precise  estimation  of  tendencies  toward  particular 
relationship  structures  as  well  as  behavioral  and  attitudinal  predictions.  However,  both  classes  of 
models  also  require  comparatively  detailed  data  about  individuals  and  their  relationships  and  thus 
demand  more  data  than  Blau  space  models.  Next,  we  address  cognitive  models  and  explore 
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decision-making  processes.  These  models  focus  on  individual,  rather  than  group,  behavior  and 
therefore  can  be  relatively  data  intensive.  Using  these  models  with  social  network  and  influence 
models  can  yield  valuable  insights,  but  further  increases  the  data  required.  Finally,  we  discuss  game 
theory  and  promulgame  models,  which  direct  attention  toward  the  individuals  and  groups 
evaluated  in  the  earlier  approaches,  the  features  of  the  environment,  and  the  interactions  between 
them.  Thus,  the  first  main  section  of  the  chapter  progresses  from  high-level  abstractions  through 
increasingly  detailed  and  data-intensive  models  before  concluding  with  approaches  that  deal  with 
situational  context. 

We  then  describe  the  gaps  and  limitations  affecting  these  models,  thereby  revealing  some  of  the 
work  that  remains  to  be  done.  All  of  these  models  are  subject  to  ongoing  development  and  none 
has  reached  the  level  of  turn-key  solutions.  Finally,  we  explore  some  of  the  steps  that  developers 
must  take  to  transition  these  models  into  operational  systems. 

This  chapter  is  intended  to  be  useful  to  readers  at  a  wide  variety  of  levels  of  expertise,  ranging 
from  novice  researchers  to  field  practitioners.  Needless  to  say,  it  is  impossible  to  write  a  chapter 
that  meets  the  information  needs  of  all  potential  audiences,  so  portions  of  the  content  may  be  too 
technical  for  some  readers  and  too  simple  for  others.  Thus,  readers  should  take  this  chapter  as  an 
introduction  that  guides  them  through  some  of  the  most  promising  areas  of  research  and  gives  a 
flavor  of  what  these  approaches  can  do.  Sources  listed  in  the  bibliography  provide  more  technical 
detail. 


2.  State  of  the  Art 
2.1.  Blau  Space  Models 

Blau  space,  named  for  Peter  M.  Blau,  is  a  /c-dimensional  system  in  which  each  dimension  is  a 
sociodemographic  variable  and  each  point  is  represented  by  a  set  of  k  coordinates  (McPherson, 
1983;  McPherson  &  Ranger-Moore,  1991).  Each  set  of  coordinates  represents  a  unique 
combination  of  sociodemographic  characteristics.  Both  individuals  and  organizations  can  be 
represented  in  Blau  space  at  locations  that  correspond  to  these  combinations.  In  other  words,  Blau 
space  represents  combinations  of  variables  such  as  age,  education  level,  and  wealth  as  locations 
within  a  multidimensional  space. 

This  multidimensional  space  conveys  information  because  of  homophily  (Lazarsfeld  &  Merton, 
1954),  or  the  tendency  for  individuals  to  associate  with  those  similar  to  themselves  (e.g.,  "birds  of  a 
feather  flock  together").  Homophily  is  one  of  the  most  robust  of  all  social  science  findings  (e.g., 
McPherson,  Smith-Lovin,  &  Cook,  2001)  and  plays  a  significant  role  in  determining  association  in 
both  the  United  States  (Marsden,  1987,  1988)  and  elsewhere  (Brashears,  2008).  As  a  consequence 
of  homophily,  nearness  in  a  Blau  space  model  is  positively  related  to  the  likelihood  of  association. 
Blau  space  therefore  allows  us  to  use  demographic  similarity  to  yield  a  probabilistic  depiction  of 
interpersonal  ties,  allowing  analysts  to  produce  an  overall  picture  of  population-level  social 
networks  without  the  need  to  collect  expensive  network  data. 
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In  addition  to  capturing  individual  networks,  Blau  space  permits  the  modeling  of  ecological 
competition  between  organizations  and  cultural  products  (e.g.,  musical  styles).  All  organizations 
require  members  and  individuals  are  more  likely  to  join  an  organization,  or  adopt  a  cultural 
product,  if  their  associates  have  already  done  so.  Even  in  the  labor  market  network  recruitment 
(e.g.,  referrals)  accounts  for  approximately  half  of  all  hires  (Bewley,  1999;  Granovetter,  1995; 
Marsden  &  Gorman,  2001).  Because  individuals  join  groups  to  which  their  associates  belong,  and 
associates  are  typically  homophilous  (i.e.,  similar  in  multiple  demographic  attributes),  organizations 
recruit  from  a  limited  and  contiguous  area  of  Blau  space  known  as  a  "niche/'  Because  both 
individual  time  and  the  number  of  persons  in  a  given  segment  of  a  population  (e.g.,  a  niche),  are 
limited,  organizations  whose  niches  overlap  compete  for  the  same  finite  pool  of  resources  (i.e., 
member  time).  The  larger  the  number  of  organizations  that  attempt  to  draw  members  from  the 
same  area  in  a  Blau  space  model  (i.e.,  the  same  population  segment),  the  more  competition  these 
organizations  experience. 

The  degree  of  competition  in  a  region  of  Blau  space  has  predictable  effects  on  organizational 
memberships.2  In  areas  of  high  competition,  current  members  are  more  likely  to  leave  the 
organization  (McPherson  &  Rotolo,  1996;  Popielarz  &  McPherson,  1995)  and  recruitment  efforts 
will  be  less  successful  as  rival  organizations  pursue  the  same  individuals.  In  contrast,  areas  of 
relatively  low  competition  produce  more  successful  recruitment  and  less  attrition.  As  a  result,  over 
time  niches  tend  to  move  away  from  areas  of  high  competition  and  towards  areas  of  low 
competition. 

Competing  organizations  often  differ  in  their  degree  of  specialization,  which  has  important 
consequences  for  the  competitive  ecology.  For  example,  a  chess  club  may  occupy  only  a  small 
niche,  while  a  church  occupies  a  broad  niche.  To  the  extent  that  their  niches  overlap,  these 
organizations  compete  for  the  time  and  attention  of  the  same  people.  Competition  is  not 
symmetric,  however,  and  the  extent  of  the  church's  niche  may  mean  that  it  only  overlaps  slightly 
with  that  of  the  chess  club,  whereas  the  church  entirely  encompasses  the  chess  club's  niche.  Thus, 
the  chess  club  competes  with  the  church  for  all  of  its  members  while  the  church  only  competes 
with  the  chess  club  for  a  few  of  its  members.  A  multi-organization  system  reaches  equilibrium 
when  the  full  carrying  capacity  (i.e.,  quantity  of  potential  members)  is  allocated  to  the 
organizations  in  a  manner  determined  by  their  degree  of  mutual  competition. 

While  the  organizational  ecology  can  reach  equilibrium,  periodic  changes  in  the  carrying  capacity 
generate  shifts  in  the  positions  of  organization  niches  and  prevent  a  static  equilibrium  (McPherson, 
2004).  However,  the  direction  of  niche  movement  can  be  predicted,  allowing  analysts  to  assess  the 
future  composition  of  a  group  (Bonikowski,  2010;  McPherson  &  Ranger-Moore,  1991).  Thus,  Blau 
space  permits  us  to  construct  an  ecological  model  of  organizational  membership  and  to  predict  the 
changing  demographics  of  organizations— at  least  in  part— through  a  competitive  process  of 
recruitment  and  retention. 


2  Organizations  need  not  realize  that  they  are  competing.  As  long  as  two  organizations  exploit  the  same  finite 
pool  of  resources,  they  are  in  fact  competing. 
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The  affiliational  ecology  model  was  originally  constructed  to  explain  the  membership 
characteristics  of  organizations,  but  has  also  been  applied  to  cultural  products  (Mark,  2003)  such  as 
musical  styles  (Mark,  1998),  including  change  over  time  (Bonikowski,  2010).  In  other  words, 
cultural  products,  such  as  musical  styles,  can  be  modeled  as  competing  for  supporters  in  much  the 
same  way  that  organizations  compete  for  members.  In  theory,  a  wide  variety  of  phenomena  can  be 
modeled  in  a  similar  fashion  (McPherson,  2004). 

McPherson  (1983)  first  proposed  Blau  space  models,  and  this  approach  continues  to  develop. 
Recent  research,  funded  by  the  Defense  Threat  Reduction  Agency,  has  continued  to  elaborate  and 
refine  the  basic  model.  This  stream  of  research  has  shown  that  models  using  Blau  space  methods 
can  predict  both  deviant  and  conventional  behaviors  at  the  individual  level  (Brashears,  Genkin,  & 
Suh,  2012;  Suh,  Brashears,  &  Genkin,  2012).  Importantly,  organizational  competition  also 
influences  the  behavior  of  organization  non-members  (Brashears,  Genkin,  &  Suh  2012;  Suh, 
Brashears,  &  Genkin,  2012).  In  combination  with  earlier  work,  this  suggests  the  possibility  of 
predicting  individuals'  tendencies  to  engage  in  various  types  of  behaviors  (e.g.,  suicide  bombing) 
using  an  inexpensive  Blau  space  approach. 

Additional  research  seeks  to  generalize  the  concept  of  the  niche  to  individuals.  In  this  model,  an 
individual's  Blau  space  neighbors  represent  a  set  of  others  who  have  regular  contact  with  that 
person  and  thus  exert  consistent  influence,  regardless  of  whether  they  are  directly  connected  to 
the  individual.  Such  others  often  share  a  physical  or  social  context  with  the  individual,  or  are 
connected  via  third  parties,  and  thus  serve  as  both  a  reference  group  and  a  measure  of  the  local 
informational  environment.  This  method  has  proven  effective  in  predicting  health  perceptions  and 
behaviors  in  a  U.S.  sample,  even  when  controlling  for  a  wide  variety  of  covariates  and  direct 
network  effects  (Behler,  Suh,  Brashears,  &  Shi,  2013).  Thus,  Blau  space  models  offer  a  powerful  and 
flexible  way  to  model  human  behavior  using  minimal  data,  but  generally  produce  a  high-level 
abstraction  of  the  modeled  social  system. 

2.2.  Models  for  Social  Networks 

Human  social  networks  are  of  increasing  interest  to  scholars  and  analysts,  but  drawing  valid 
statistical  inferences  about  these  networks  poses  challenges.  Most  statistical  techniques  assume 
that  observations  are  independent  (e.g.,  the  opinion  of  one  respondent  to  a  survey  does  not 
determine  the  opinion  of  another  respondent),  but  this  assumption  cannot  apply  to  social 
networks.  Individuals  who  share  a  relationship  (i.e.,  are  tied)  influence  one  another,  and  thus 
standard  statistical  methods  cannot  be  employed  (e.g.,  Marsden  &  Friedkin,  1993).  Additionally, 
network  data  are  expensive  to  collect  and  few  analysts  have  access  to  a  "sample"  of  networks.  As  a 
result,  analysts  cannot  use  standard  techniques  to  compute  statistically  valid  estimates  of  network- 
level  processes.  Both  of  these  difficulties  are  exacerbated  when  dealing  with  data  collected  over 
time,  which  allows  more  certain  identification  of  causal  relationships,  but  also  introduces  statistical 
dependence  on  a  longitudinal  as  well  as  a  cross-sectional  basis.  Finally,  analysts  often  wish  to 
explore  how  network  structures  (e.g.,  who  is  friends  with  whom)  and  actor  attributes  (e.g., 
characteristics  and  behaviors)  influence  each  other,  thereby  creating  even  more  complex 
dependency  structures.  For  example,  are  individuals  influenced  to  become  suicide  bombers  by 
their  associates,  or  do  individuals  who  want  to  become  suicide  bombers  tend  to  become 
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associates?  Fitting  models  to  network  data  calls  for  specialized  techniques,  and  two  types  of 
models  have  gained  considerable  traction:  Exponential  Random  Graph  Models  (ERGMs),  which 
parameterize  graph  features,  and  Siena  models,  which  parameterize  the  tendencies  of  simulated 
actors. 

ERGMs  begin  with  the  observation  that  any  realized  network  is  a  specific  instance  drawn  from  a 
population  of  potential  networks.  For  example,  a  group  of  15  individuals  can  have  a  total  of  n(n-l) 
=  15(15-1)  =  210  relationships.  If  these  relationships  are  binary  (i.e.,  a  relationship  either  exists  or 
does  not  exist),  then  the  sample  space  from  which  the  observed  network  is  drawn  consists  of  2210 
or  1.64  x  1063  possible  networks.  The  probability  of  observing  any  specific  network  in  this  space, 
assuming  all  are  equally  likely,  is  1/2210  (Wasserman  &  Faust,  1997).  However,  while  all  networks 
might  be  equally  likely  to  occur,  certain  network  characteristics  are  more  likely  to  occur  than 
others.  For  example,  only  one  network  contains  all  210  possible  ties  and  only  one  network  contains 
0  ties,  but  many  networks  contain  105  ties.3  Thus,  models  can  generate  expectations  for  how  many 
ties  a  network  drawn  at  random  from  this  sample  space  should  contain.  If  the  observed  network 
contains  many  more  or  many  fewer  ties  than  this  expected  value,  we  can  infer  that  some 
mechanism  is  encouraging,  or  discouraging,  the  formation  of  social  ties.  The  same  logic  extends  to 
other  network  processes,  including  reciprocity  (i.e.,  the  tendency  for  individuals  to  select  each 
other  as  associates)  and  transitivity  (i.e.,  the  friend  of  my  friend  is  my  friend),  among  others. 
ERGMs  estimate  parameters  describing  how  much  more  (or  less)  often  a  particular  feature  is 
observed  than  we  would  expect  by  chance. 

The  statements  above  require  two  elaborations.  First,  because  many  characteristics  of  a  network 
become  more  probable  when  many  ties  exist  (e.g.,  reciprocity  is  higher  when  there  are  more  ties 
overall),  parameter  estimation  must  control  for  the  number  of  ties  in  the  network.  ERGMs  impose 
this  control  by  limiting  the  sample  space  to  those  graphs  with  the  same  number  of  nodes  (i.e., 
vertices)  and  ties  (i.e.,  edges),  as  the  observed  graph.  Second,  while  the  ties  are  binary,  the 
probability  that  a  tie  exists  does  not  have  to  be  0.50.  Applying  a  Bernoulli  distribution  can  model 
the  probability  that  a  tie  exists  across  the  potential  range,  and  these  probabilities  can  be 
conditioned  on  other  factors,  including  the  size  and  density  of  the  network  as  well  as  the  attributes 
of  actors  and  actor  pairs,  yielding  a  set  of  conditional  uniform  distributions. 

The  general  ERG  model  (Lusher,  Koskinen,  &  Robins,  2013;  Robins,  Pattison,  Kalish,  &  Lusher,  2007) 
parameterizes  particular  network  features.  One  feature  might  be  a  closed  triad  (i.e.,  a  group  of 
three  nodes  that  all  have  connections  to  each  other)  while  another  might  be  an  open  triad  (i.e.,  a 
group  of  three  nodes  that  contains  only  two  ties).  The  model  calculates  statistical  coefficients 
representing  the  tendency  for  a  given  graph  to  contain  more,  or  fewer,  of  these  features  than 
would  be  expected  by  chance.  It  determines  parameters  using  Monte  Carlo  Markov  Chain 
Maximum  Likelihood  Estimation  techniques,  which  permit  inclusion  of  a  very  large  variety  of 


3  To  use  a  more  familiar  example,  when  rolling  two  (fair)  six-sided  dice,  only  one  combination  sums  to  2  (1+1) 
and  one  combination  sums  to  12  (6+6),  but  six  combinations  sum  to  7  (1+6,  6+1,  2+5,  5+2,  3+4,  4+3).  Thus, 
even  though  all  possible  combinations  are  equally  likely,  combinations  that  sum  to  7  are  much  more  likely  to 
occur  than  combinations  summing  to  2  or  12. 
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configurations  (Snijders,  Pattison,  Robins,  &  Handcock,  2006;  Wasserman  &  Robins,  2005), 
although  estimating  parameters  with  larger  numbers  of  nodes  and  ties  can  consume  considerable 
time.  These  techniques  generate  a  set  of  graphs  using  randomized  starting  values  for  the  model 
parameters  and  compare  them  to  the  observed  graph;  if  the  observed  graph  has  a  very  low 
likelihood  of  belonging  to  this  set,  the  model  selects  new  parameter  values  and  repeats  the 
process.  Over  many  replications,  the  process  homes  in  on  the  parameter  values  most  likely  to  have 
produced  the  observed  graph.  Modelers  must  exercise  care,  however,  to  ensure  that  parameter 
estimates  remain  stable  against  local  maxima  (i.e.,  models  should  be  estimated  several  times  with 
different  initial  seeds).  They  may  also  have  to  include  some  configurations  in  the  model  regardless 
of  theory  in  order  to  avoid  near  degeneracy,  or  a  situation  in  which  only  a  small  subset  of  all 
possible  graphs  has  other  than  a  very  low  probability  (Handcock,  2003).  A  good  fitting  model  can 
be  defined  as  one  that  yields  a  very  high  likelihood  of  obtaining  the  observed  graph  relative  to  the 
alternatives. 

Scholars  continue  to  develop  the  ERGM  framework.  Wang,  Robins,  Pattison,  and  Lazega  (2013) 
have  generalized  ERGMs  to  accommodate  two-level  scenarios  (e.g.,  individuals  nested  within 
groups).  This  approach  resembles  hierarchical  linear  modeling  and  allows  analysts  to  examine 
interactions  between  individual  relations  and  higher  level  structures  (e.g.,  organizations).  However, 
the  software  implementation  of  this  model  is  currently  somewhat  unreliable.  Other  work  (Caimo  & 
Friel,  2012)  has  wedded  ERGMs  to  a  Bayesian  framework  to  assess  the  quality  of  model  fit.  This 
helps  to  avoid  reliance  on  frequentist  model  fit  measures,  which  depend  upon  as  yet  poorly 
understood  comparison  distributions.  Thus,  in  addition  to  considerable  empirical  work  using 
ERGMs,  the  method  continues  to  acquire  methodological  sophistication. 

ERGMs  capture  social  processes  by  parameterizing  features  of  the  network,  but  impose  no 
assumptions  about  the  mechanisms  that  generate  these  features.  This  adds  to  the  model's 
flexibility,  but  can  leave  the  analyst  in  the  position  of  trying  to  justify  configurations  that  lack  a 
clear  theoretical  meaning.  Stochastic  actor-based  models  for  network  dynamics  (i.e.,  Siena  models) 
avoid  this  problem  by  building  stronger  assumptions  into  the  model.  Additionally,  Siena  models  are 
ideal  for  longitudinal  data. 

A  Siena  model  represents  interpersonal  ties  as  states  rather  than  events,  meaning  that  it  shows  any 
particular  communication  event  (e.g.,  an  email  exchange)  simply  as  an  expression  of  an  underlying, 
ongoing  relationship.  States  can  change  (i.e.,  ties  form  and  are  dissolved)  but  in  general  existing 
ties  tend  to  continue  existing  and  nonexistent  ties  tend  to  continue  not  existing.  Time  varies 
continuously  in  arbitrarily  defined  segments  and  actors  control  their  outgoing  ties  (i.e.,  individuals 
choose  whom  they  prefer)  but  not  their  incoming  ties  (i.e.,  individuals  cannot  choose  who  prefers 
them).  Finally,  Siena  models  employ  simulated  actors,  or  simplified  mathematical  representations 
of  the  entities  (e.g.,  individuals)  that  make  up  the  nodes  of  the  network. 

A  Siena  model  proceeds  via  two  mechanisms:  the  change  opportunity  process  and  the  change 
determination  process.  The  change  opportunity  process  determines  the  rate  at  which  actors  in  the 
model  have  the  opportunity  to  alter  their  network  ties  or  behaviors.  In  its  simplest  form,  this  rate  is 
uniform  across  all  actors,  meaning  that  at  any  given  time  all  actors  are  equally  likely  to  make  a 
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discrete  change  (i.e.,  all  individuals  change  at  the  same  rate).  Alternatively,  the  rate  can  be 
conditional,  meaning  that  some  actors  (e.g.,  more  popular  individuals)  are  more  likely  to  make 
discrete  changes  than  others  (i.e.,  exhibit  a  higher  rate  of  change).  The  rate  of  change  can  also  vary 
over  time  (e.g.,  change  occurs  at  an  increasing  rate).  The  model  determines  stochastically  which 
actor  will  make  a  change,  with  the  rate  controlling  the  probability  that  any  given  actor  will  be 
chosen. 

The  change  determination  process  models  the  changes  that  the  actors  make  when  given  the 
chance.  Actors  may  make  a  single  change  per  opportunity:  add  a  single  tie,  eliminate  a  single  tie, 
change  one  attribute,  or  do  nothing.  This  one-change  rule  defines  a  finite  set  of  alternative 
networks  that  actors  can  reach  at  the  present  time.  The  actor's  objective  function,  which  defines 
how  rewarding  particular  network  structures  are  for  the  individual  actor,  determines  which  change 
is  made  (i.e.,  which  alternative  network  is  selected).  Put  simply,  the  actor  will  tend  to  select  the 
alternative  that  produces  the  most  positive  objective  function.  However,  because  the  model  is 
stochastic,  actors  do  not  always  select  the  change  that  yields  the  greatest  improvement  in  the 
objective  function.  Thus,  while  the  objective  function  can  be  thought  of  in  terms  of  "p references/' 
it  is  more  accurate  to  view  it  as  capturing  tendencies  stemming  from  preferences  as  well  as 
circumstances  beyond  individual  control  (e.g.,  the  availability  of  certain  types  of  others  in  the 
population). 

The  effects  in  a  Siena  model  refer  to  the  tendencies  of  simulated  actors  to  make  specific  types  of 
changes  (Snijders,  Van  de  Bunt,  &  Steglich,  2010).  The  effects  can  be  defined  by  structure  (e.g., 
reciprocity,  transitivity),  by  attributes  (e.g.,  level  of  behavior  adoption),  as  well  as  interactions 
between  the  two  (e.g.,  homophily). 

The  Siena  approach  uses  a  method-of-moments  procedure  to  generate  parameter  estimates  and 
therefore  models  the  process  of  change  between  time  points  rather  than  tendencies  at  any  one 
time.  The  first  time  point  is  not  modeled,  serving  instead  as  a  start  for  the  estimation  process.  As  a 
result,  a  Siena  model  cannot  be  estimated  without  data  from  at  least  two  points  in  time. 

In  outline,  the  Siena  model  proceeds  by  taking  the  first  wave  of  data  (t)  as  the  starting  point  for  the 
simulation  model.  An  actor  is  chosen  at  random,  with  the  probability  determined  by  the 
parameters  of  the  change  opportunity  rate  (uniform  or  conditional).  The  model  computes  the 
objective  function  for  both  their  current  network  structure  as  well  as  all  network  structures 
reachable  via  only  a  single  change,  with  the  probability  of  a  given  change  determined  by  the  size  of 
each  objective  function,  including  the  function  for  the  existing  network  structure.  Once  the  model 
has  selected  a  given  network  using  this  procedure,  the  change  (if  any)  is  made  and  a  new  actor  is 
chosen  via  the  change  opportunity  process.  Change  in  the  network  is  modeled  as  a  Markov 
process:  the  state  of  the  network  at  the  current  time  (t)  probabilistically  determines  the  future 
(M-l)  state  of  the  network,  with  no  additional  effects  felt  from  prior  (t-k)  states.  This  process 
repeats  many  times,  in  theory  yielding  outcome  networks  comparable  to  the  observed  networks  at 
later  time  points  (i.e.,  M-l,  t+2,  etc.). 

Initially  the  analyst  sets  the  parameters  in  the  objective  function  randomly  and  the  software 
compares  the  performance  of  the  simulation  to  the  observed  data.  If  the  simulation  fares  poorly 
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the  values  are  adjusted  and  the  simulation  process  is  repeated.  Eventually,  the  model  homes  in  on 
the  best  set  of  parameter  values  (including  those  for  the  rate  of  change  parameters)  for 
reproducing  the  observed  networks.  As  with  ERGMs,  however,  modelers  must  take  care  to  ensure 
that  parameters  have  not  been  trapped  in  local  maxima;  re-estimating  with  new  seed  values  is 
advised. 

Recent  elaborations  of  the  Siena  model  have  explored  its  applicability  to  two-mode  data  (Snijders 
et  al.,  2012),  allowing  researchers  to  connect  individual  and  organizational  processes  in  a  single 
longitudinal  model.  Brandes,  Indlekofer,  and  Mader  (2012)  have  also  developed  diagnostic 
visualization  tools  that  assist  researchers  in  determining  the  quality  of  model  fit.  Thus,  the  Siena 
approach  is  still  undergoing  considerable  development. 

Siena  models  and  ERGMs  differ  in  their  overall  perspective  on  social  networks.  Siena  models  view 
the  actors  as  making  changes  to  their  networks  in  pursuit  of  particular  structures.  This  has  the 
advantage  of  producing  parameters  that  allow  many  small  changes  to  a  network  to  accumulate 
into  overall  network  configurations,  but  runs  the  risk  of  treating  actors  as  having  more  control  over 
the  social  system  than  they  really  do.  ERGMs,  in  contrast,  focus  on  tendencies  for  particular 
features  to  appear  more  often  than  expected  by  chance,  but  make  no  effort  to  model  the 
processes  that  give  rise  to  these  features.  Therefore,  analysts  can  identify  features  that  are 
preferred  more  often  than  others,  but  that  cannot  be  realized  through  any  straightforward 
process.  The  model  preferred  for  a  given  application  should  depend  on  data  availability,  on 
whether  the  data  are  longitudinal  or  cross-sectional,  and  on  whether  the  nodes  in  the  network  can 
reasonably  be  thought  of  as  "agents"  (e.g.,  transformers  in  a  power  grid  are  not  "agents"  in  the 
same  sense  as  human  individuals  in  a  social  network). 

2.3.  Social  Influence  Network  Theory 

To  understand  the  hearts  and  minds  of  a  population  we  must  study  attitudes.  Social  Influence 
Network  Theory  (SINT)  fills  this  gap  by  modeling  valued  attitudes,  meaning  affective  (i.e., 
emotional)  views  of  some  issue  that  can  range  from  very  positive  to  very  negative.  The  theory 
assumes  that  individuals  enter  an  interaction  with  some  initial  attitude  and  are  susceptible  to 
influence  from  others.  The  degree  of  influence  is  determined  by  three  characteristics:  the  network 
of  relations  among  individuals,  the  weights  placed  on  these  relations,  and  the  attitude  of  each 
relationship  partner. 

First,  individuals  are  tied  to  others  in  a  social  network.  We  exchange  information,  views,  and 
camaraderie  with  our  associates,  and  their  attitudes  can  influence  our  own  views.  Additionally, 
however,  our  associates  have  relationships  to  others  to  whom  we  are  not  connected  (i.e.,  our 
second-order  network).  As  those  others  influence  the  attitudes  of  our  associates  they  will, 
indirectly,  influence  our  own  attitudes.  To  understand  the  process  of  attitude  change  and 
consensus,  we  must  model  the  entire  network. 

Second,  we  do  not  necessarily  listen  to  each  of  our  associates  equally  carefully.  Some  of  them  have 
a  strong  influence  on  our  own  views,  while  we  may  tend  to  ignore  others.  As  a  result,  the  model 
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must  take  account  of  the  unequal  influence  of  various  associates,  and  must  factor  in  our  relative 
level  of  self-confidence. 

Finally,  the  extent  to  which  initial  attitudes  must  change  in  order  to  reach  consensus  is  at  least 
somewhat  a  function  of  starting  positions.  Individuals  who  begin  relatively  further  apart  on  an  issue 
must  change  more  in  order  to  reach  consensus  than  those  who  begin  near  each  other. 

The  combination  of  these  straightforward  elements  yields  complex  and  interesting  behavior,  as 
each  can  moderate  or  enhance  the  impact  of  the  others.  For  example,  very  different  initial 
positions  may  result  in  relatively  little  attitude  change  if  few  direct  network  connections  exist 
between  opposite  extremes  and  if  the  self-weights  are  very  high.  Alternatively,  when  others  are 
very  influential,  attitudes  may  change  so  rapidly  that  the  system  will  repeatedly  overshoot 
consensus,  ultimately  requiring  longer  to  reach  equilibrium.  Therefore,  while  the  model  is 
conceptually  simple,  it  has  rich  implications. 

The  SINT  model  (Friedkin  &  Johnsen,  2011)  includes  several  important  elements.  First,  an  actor's 
self-confidence  is  locked  in  a  zero-sum  game  with  the  influence  of  others;  actors  less  certain  of 
their  own  views  are  more  susceptible  to  the  attitudes  of  others.  Second,  all  persons  to  whom  an 
actor  is  tied  have  an  influence  on  the  actor's  future  views,  with  the  degree  of  influence  determined 
by  the  product  of  their  level  of  influence  and  their  current  attitude.  Third,  the  views  of  the  actor  at 
the  first  time  point  (t=l)  exert  a  continuing  influence  in  the  model  at  all  points  in  time.  This  last 
element  is  extremely  useful  in  preventing  the  model  from  predicting  an  unrealistic  degree  of 
consensus,  but  also  treats  the  first  time  point  as  special  relative  to  the  remaining  points.  In  fact, 
because  the  analyst  chooses  the  first  time  point  based  on  convenience  or  data  availability,  its 
theoretical  value  is  dubious.  Finally,  the  model  is  computed  iteratively;  each  computed  set  of 
attitudes  is  reentered  to  predict  the  attitudes  at  the  next  time  step  until  the  system  reaches 
equilibrium. 

SINT  allows  an  analyst  to  predict  the  equilibrium  states  of  members,  but  two  or  more  consensus 
positions  are  possible  at  equilibrium,  implying  some  ongoing  disagreement.  In  all  but  a  very  few 
boundary  cases  the  equilibrium  consensus  positions  lie  on  the  convex  hull  of  the  initial  positions.4 
Put  differently,  the  equilibrium  consensus  will  not  be  more  extreme  along  any  attitude  dimension 
than  the  most  extreme  initial  positions  in  those  dimensions.  The  model  has  an  additional 
advantage  in  that  it  can  derive  many  of  the  quantities  needed  from  partial  information.  For 
example,  if  data  on  initial  and  ending  attitudes  are  available,  as  well  as  network  data  on  the  actors, 


4  In  dyads  where  both  partners  heavily  influence  the  other,  consensus  positions  sometimes,  though  rarely, 
appear  outside  the  convex  hull.  This  appears  to  result  from  an  attempt  to  escape  from  a  coordination 
problem.  In  these  systems  interaction  partners  repeatedly  overshoot  each  other  in  an  attempt  to  reach 
agreement,  and  so  one  partner  adopts  a  more  extreme  position  in  order  to  provide  a  conspicuous  target  for 
the  other.  For  example,  a  married  couple  might  try  to  choose  a  restaurant  for  dinner.  Each  suggests  a 
restaurant  and  each  attempts  to  defer  to  the  other's  preference,  thus  preventing  consensus  and  delaying  the 
meal.  One  partner  might  attempt  to  break  the  cycle  by  suggesting  a  third  option  that  usually  neither  prefers. 
While  this  example  is  only  loosely  accurate  (i.e.,  the  example  deals  with  a  decision  rather  than  an  attitude),  it 
does  help  illustrate  the  general  process. 
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the  model  can  derive  the  weights  on  those  relationships.  Thus  SINT  provides  a  flexible  and  effective 
way  to  model  attitude  change  and  stability  in  groups. 

2.4.  Cognitive  Models 

Researchers  have  used  a  variety  of  mathematical  and  computational  models  to  emulate  the 
cognitive  process.  The  examples  summarized  below  highlight  some  of  the  most  influential. 

An  early  example  focused  on  Belief,  Desire,  and  Intention  (BDI).  Emerging  from  artificial 
intelligence,  the  BDI  model  treats  the  three  constitutive  concepts  as  key  pragmatic  dimensions  of 
cognition  (Bratman,  1987).  Belief  encompasses  those  assertions  that  the  actor  considers  true  and 
that,  in  some  cases,  act  as  a  placeholder  for  the  actor  ontology  (although  the  actor  may  not  be 
consciously  aware  of  them).  Desire  conveys  focus  and  is  sometimes  used  interchangeably  with 
goals.  Intention  describes  the  phase  in  which  belief  and  desire  have  congealed  into  a  settled  plan, 
or  course  of  action. 

BDI  models  haves  been  criticized  for  doing  both  too  much  and  too  little.  For  example,  some  critics 
say  that  having  incorporating  only  three  steps  makes  the  mechanism  underlying  BDI  models  too 
simple,  while  others  view  three  interacting  dimensions  as  overkill  (Rao  &  Georgeff,  1995). 
However,  BDI  models  suffer  from  their  ultimate  limitation  in  that,  while  seemingly  intuitive,  they 
have  no  obvious  foundation  in  either  decision  theory  or  substantive  social  action  that  would  shape 
and  drive  their  computational  implementation  and/or  the  resulting  dynamics.  Such  criticism 
suggests  that  BDI  is  constructed  on  the  basis  of  folk  psychology  and,  thus,  cannot  provide  models 
with  more  than  a  pseudo-cognitive  placeholder. 

The  rational  choice  model  of  cognition,  as  developed  by  Becker  (1976),  Coleman  (1990),  and 
numerous  others  (cf.,  Elster  1986),  has  played  an  influential  role  in  the  social  sciences,  albeit 
arousing  controversies  along  the  way  (Coleman  &  Fararo,  1992;  Green  &  Shapiro,  1994).  The 
formulation  of  individual  models  depends  on  the  domain  and  the  research  strategy,  but  all  rational 
choice  models  provide  a  mechanism  for  assessing  the  costs,  benefits,  and  tradeoffs  of  different 
courses  of  action,  and  the  responses  that  result  from  such  assessments. 

One  strength  of  rational  choice  lies  in  its  high  level  of  generalizability.  Researchers  have  used  the 
paradigm  in  a  wide  range  of  social  domains,  from  microeconomics  (Field,  1984)  to  international 
relations  (Levy,  1997).  Often  researchers  have  been  able  to  identify  rationalities  underlying 
strategies  that,  on  the  surface,  appeared  to  be  irrational  or  self-defeating  (Pape,  2005). 

However,  a  primary  weakness  of  rational  choice  is  that  it  typically  involves  closed  concepts  of  gain 
and  loss.  Modelers  often  act  as  if  these  concepts  are  self-evident  and,  as  a  result,  many  models 
weakly  represent  the  dynamic  spectrum  of  priorities  that  social  actors  embrace,  including  their 
evolution  over  time.  Another  problematic  characteristic  concerns  how  the  models  represent 
emotion  as  a  factor  in  decision  processes.  Often  they  simply  ignore  affect  and,  when  they  do 
address  emotions,  frequently  view  them  primarily  as  a  source  of  irrationality  (Doran,  2000).  This 
undermines  possible  analysis  of  strongly  committed  action  and/or  the  complementary  ways  in 
which  emotion  and  cognition  work  to  achieve  insights  and  effective  action. 
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Randall  Collins  has  been  a  major  contributor  to  cognitive  models  in  the  social  sciences  that 
integrate  both  emotion  and  ritual,  creating  a  more  extended  concept  of  cognition.  He  developed 
his  theory  of  emotion  in  social  life  over  a  period  of  time  (Collins,  1981a,  1981b,  1990;  Kemper  & 
Collins,  1990),  ultimately  proposing  a  mechanism  designed  to  subsume  rational  choice  models 
(1993). 

Collins'  incorporation  of  emotion  into  sociological  theory  is  based  upon  Durkheim's  discussion  of 
religious  ritual  (1995)  and  Goffman's  subsequent  analysis  of  interaction  rituals  (1967),  as  unified  in 
his  own  theory  of  interaction  ritual  chains  (IRCs)  (Collins,  1981a,  1981b,  1990,  2005).  In  brief, 
Collins  argues  that  group  solidarity  at  all  levels  is  grounded  in  emotion,  that  emotional  bonds 
precede  and  make  possible  coercive  and  contractual  relations,  and  that  IRCs  of  widely  varying 
types  generate  group  solidarity  as  well  as  emotional  energy  (EE)  among  the  participants.  The 
efficacy  of  specific  IRCs  varies  depending  on  the  attributes  of  group  interaction:  physical  assembly, 
boundedness,  focus  of  attention  and  awareness,  commonality  of  emotional  mood,  and  symbols 
that  represent  group  membership  (Collins,  1988;  Collins,  1993). 

Collins  (1993)  presented  IRC/EE  as  a  common  metric  and  motivation  underlying  all  social 
exchanges.  His  model  recognizes  "emotional  solidarity  with  a  group  as  the  primary  good  in  social 
interaction"  (1993,  p.  205).  He  notes  that  individuals  "are  motivated  to  maximize  the  amount  of 
solidarity  they  can  receive,  relative  to  the  costs  of  producing  it"  (1993,  p.  209). 

Nonetheless,  Collins's  IRC/EE  model  has  weaknesses  that  prevent  realization  of  its  putative 
advantages.  First,  contrary  to  Collins's  assertions,  there  is  little  reason  to  believe  that  the 
interchange  of  interaction  rituals  fully  determines  economic  markets.  Second,  serious  questions 
arise  as  to  whether  IRC  markets  operate  in  the  narrow  way  that  the  model  suggests.  However, 
Collins's  model  has  served  as  a  springboard  to  a  more  general  affect-based  field  theory  (Sallach, 
2008)  that  avoids  the  limitations  of  the  IRC/EE  model  in  its  speculative  1993  formulation  while,  at 
the  same  time,  integrating  emotional  and  cognitive  processes  for  social  actors  at  multiple  scales. 
This  integration  allows  behavior  previously  regarded  as  generically  "irrational"  to  more  specifically 
recognize  the  underlying  emotional  commitments  that  override  utilitarian  considerations, 
sometimes  in  a  strongly  persistent  manner  (cf.,  Benmelech  &  Berrebi,  2007). 

2.5.  Game  Theoretic  Models 

The  formalisms  most  frequently  associated  with  adversarial  interaction  derive  from  game  theory. 
They  have  been  applied  widely,  and  are  one  of  the  rare  forms  of  social  analysis  that  can  explore 
decision  processes  and  outcomes  using  deductive  analysis,  while  increasingly  supporting  advanced 
simulation  environments  as  well  (cf.,  Parsons,  Gmytrasiewicz,  &  Wooldridge,  2002).  Fudenberg  and 
Tirole  (1987)  summarize  the  several  sources  of  game-theoretical  strength: 

[Game  theory]  imposes  some  discipline  on  theoretical  thinking.  It  forces  economists  to  clearly 
specify  the  strategic  variables,  their  timing  and  the  information  structure  faced  by  firms  .  .  . 
[T]he  researcher  learns  as  much  from  constructing  the  model  (the  "extensive  form")  as  from 
solving  it  because  in  constructing  the  model  one  is  lead  to  examine  its  realism,  (p.  176) 
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However,  it  also  appears  that  the  more  complex  and  dynamic  the  social  phenomena  under  study, 
the  more  difficult  it  is  to  achieve  effective  game-theoretic  representation.  Peltzman  (1991),  for 
example,  bemoans  the  seeming  inability  of  theory,  especially  game  theory,  to  lead  to  powerful 
generalizations.  He  finds,  rather,  "  ...  an  almost  interminable  series  of  special  cases.  The 
conclusions  drawn  from  these  cases  tend  to  be  very  sensitive  to  the  way  problems  are  defined  and 
to  the  assumptions  that  follow"  (p.  206).  Peltzman  continues:  "By  suitably  permuting  and 
combining  the  problems  and  assumptions,  new  models  can  be  produced  almost  ad  libitum.  Indeed, 
the  production  of  new  models  and  the  tidying  up  of  old  ones  seem  to  be  major  goals  of  this 
research  enterprise"  (p.  207).  Similarly,  Fisher  (1989)  writes:  "There  is  a  strong  tendency  for  even 
the  best  practitioners  to  concentrate  on  analytically  interesting  questions  rather  than  on  ones  that 
really  matter  for  the  study  of  real-life  industries"  (p.  123). 

Game  theory  has  provided  a  mathematical  way  of  exploring  interactive  social  outcomes  under 
varying  assumptions.  However,  its  very  formalism  has  led  to  concerns  about  unrealistic 
assumptions  (e.g.,  perfect  foresight,  stylized  outcomes)  and  a  failure  to  incorporate  the  situated 
contexts  that  inform  strategic  decisions.  To  maximize  its  theoretical  and  practical  contributions  to 
social  modeling,  game-theoretic  models  need  to  become  more  dynamic.  The  resulting  models 
should  be  elegant,  and  guided  by  social  theory  and,  as  a  result,  they  will  also  be  considerably  more 
expressive. 

Human  actors  play  multiple  games,  select  among  available  games,  shift  from  one  to  another, 
sometimes  mistake  what  the  game  their  counterpart  is  playing,  act  in  ways  that  are  (more  or  less) 
effective  within  multiple  games  simultaneously,  etc.  Luttwak  (1987)  points  out  that  strategic 
choices  are  inherently  paradoxical:  because  the  strongest,  most  effective  options  are  also  the  most 
obvious  and,  thus,  are  likely  to  be  the  best  defended.  Options  that  are  suboptimal,  and  thus 
unexpected,  are  sometimes  the  most  effective.  This  emphasis  upon  surprise  is  not  just  a  product  of 
strategic  planning.  As  Miller  (1997)  has  documented,  unpredictability  has  been  essential  to  species 
survival  and  adaptive  dominance.  Accordingly,  interpretive  dynamics  that  produce  unexpected 
results  can  be  seen  to  be  essential  to  the  accomplishment  of  participatory  and  analytical 
understandings  of  strategies  and  courses  of  action. 

2.6.  Promulgames 

Promulgame  theory  represents  a  cluster  of  innovations  within  game  theory  designed  to  reduce  the 
brittleness  sometimes  found  in  formal  games  and  to  recognize  the  recurrence  and  interaction  of 
broad  game  types  (Sallach,  2006).  The  two  aspects  of  interest  are  prototypical  games  and 
mu/tigames.  Combining  the  two  types  of  games  results  in  a  promulgame ,5  which  in  turn  serves  as 
an  interactive  building  block  of  broader  sociocultural  structures  and  processes. 


5  Please  pardon  the  compound  neologism. 
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Prototype  theory  defines  the  nature  of  games  that  actors  play,  potentially  at  diverse  levels.  There 
are  three  prototypical  games:  beneficent,  instrumental,6  and  coercive,  with  gradations  of  each 
type.  Together,  they  define  a  broad  range  of  social  game  types  and  allow  exploration  of  the 
interactions  among  them.  These  games  can  be  easily  represented  by  a  unit  interval  of  [1,  -1] 
divided  into  three  ranges  of,  for  example,  [0.95  to  0.35,  0.3  to  -0.3,  and  -0.35  to  -0.95].  In  this 
example,  the  games  range  from  the  most  beneficent  to  the  most  coercive,  with  each  game  having 
an  internal  range  of  0.6. 

Promulgames  are  prototypical  in  the  sense  that  a  conceptual  core  defines  the  game  type,  while 
empirical  games  vary  in  their  proximity  to  that  core  (Rosch,  1978;  1983).  Researchers  can  identify 
and  dimensionalize  these  differences  for  modeling  purposes.  This  form  accommodates  itself  to  the 
representation  of  an  extensive  variety  of  empirical  games,  including  the  emergence  of  calibrated 
strategies. 

Beneficent  games  involve  a  kind  of  mutual  support  seen  in  families  and  tribes,  among  neighbors, 
and  within  communities.  Types  of  support  in  beneficent  games  may  vary,  but  accounting  is  not 
strict  and  the  game  tends  to  be  mutually  reinforcing  over  time,  commonly  resulting  in  virtuous 
spirals  (Carse,  1986).  Instrumental  games  are  familiar,  involving  complementary  benefit  and, 
relative  advantage,  arm's-length  accounting,  and  are  often  self-reinforcing  (Osborne  &  Rubinstein, 
1990).  Coercive  games  involve  the  exchange  (or  threat)  of  force  or  violence.  Reciprocity  is 
frequently  anticipatory,  and  comparative  accounting  tends  to  be  exaggerated,  resulting  in  a  vicious 
spiral.  Complementarity  usually  takes  the  form  of  innovative  tactics  or  novel  defenses.  Any  of  the 
three  types  of  games  can  stabilize  over  time  to  a  recurrent  balance. 

In  addition  to  reciprocal  interchanges,  where  the  details  primarily  concern  frequency,  quantity, 
and/or  quantitative  conversion,  off-diagonal  interchanges  occur  in  which  an  actor  uses  one 
resource  to  acquire  or  respond  to  a  different  one.  For  example,  actors  use  force  to  gain  material 
benefits,  or  money  to  gain  altruistic  benefits.  Therefore,  each  game  resource  requires  two 
(directional)  dimensions.  Such  games  are  apparently  less  likely  to  stabilize  than  are  reciprocal 
games,  but  are  nonetheless  interesting  by  virtue  of  their  empirical  familiarity.  For  example,  armed 
robbery  uses  coercive  means  to  acquire  instrumental  resources.  Mercenary  armies  and/or  hit  men 
use  instrumental  means  to  acquire  coercive  resources.  Revolutionaries  and  terrorists  use 
instrumental  and/or  support  services— providing  food,  welfare,  and  health  services— in  return  for 
assistance  in  conducting  coercive  acts  or  hiding  the  perpetrators.  Finally,  abducting  children  to 
raise  them  (as  opposed  to  demanding  ransom)  can  be  viewed  as  a  use  of  coercive  means  of 
acquiring  human  resources. 

Off-diagonal  games  also  figure  in  situations  characterized  by  ambiguity  or  deception  as  to  which 
games  are  being  played.  Examples  abound.  Contributions  to  charities  (support)  may  be  deceptively 
converted  to  terrorist  use  (coercion).  The  sale  of  contraband  can  be  an  economic  means  of 
achieving  the  same  result.  Therefore,  human  actors  (and,  thus,  computational  models),  need  the 


6  This  game  type  is  often  called  'economic/  but  this  characterization  is  too  narrow.  Other  forms  of 
instrumental  and/or  pragmatic  trade-offs  and  negotiations,  particularly  in  political  and  military  domains, 
provide  additional  examples  of  instrumental  games. 
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capability  to  determine  the  game  they  are  being  invited  (enticed)  to  play.  Computational  models 
must  assemble  and  parameterize  instantiated  games  endogenously  from  their  components 
(Sallach,  2000). 

Social  actors  engage  in  games  as  a  means  of  eliciting  coordination  from  other  actors  (cf.  Taylor, 
2005).  They  initiate  (or  respond  with)  games  of  different  types  based  on  various  situated  factors. 
Table  1  illustrates  two  such  factors:  a)  the  strategy  employed  by  the  actor  (top  row),  and  b)  the 
relationship  between  the  actor  and  relevant  other  actors  (bottom  row). 


Table  1.  Promulgames  by  Strategy  (Top)  and  Relationship  (Bottom) 


STRATEGY: 

Compliant  (Soft) 

Defensive  (Firm) 

Aggressive  (Hard) 

Beneficent 

Unrestrained  support; 
give  freely  without 
expectation  of  return 

Attend  to  timing  and 
adequacy  of  response;  be 
prepared  to  diminish 
reciprocity 

Withdraw  cooperation; 

shun 

Instrumental 

Relax  competition;  share 
opportunities;  tolerate 
slow  or  incomplete 

responses 

Compete;  expect  value- 
for-value;  negotiate 
binding  contracts 

Demand  hard  bargains; 

seek  to  drive 

competitor  out  of 
region  or  specialty 

Coercive 

Show  respect  for 
adversary;  accept  truces; 

allow  conflict  to  subside 

Return  blow-for-blow; 
assess  threats;  prepare 
for  possible  attacks 

Attack  without 

provocation;  amplify 
threats;  invent  reasons 
to  confront  or  destroy 

enemy 

ROLE: 

Affiliate 

Rival 

Adversary 

The  relationships  among  actors  described  in  Table  1  overlap  with  a  number  of  social  theories  and 
models.  Recognition  theory  provides  an  important  example.  Honneth  (1996)  distinguishes  among 
three  patterns  of  recognition:  love,  rights,  and  solidarity.  The  affinity  relation  that  Honneth  calls 
'love'  gives  rise  to  the  strategies  summarized  in  the  top  row,  calibrated  by  a  selected  response  to 
the  particular  situation.  The  legal  recognition  relationship  ('rights')  is  primarily  instrumental  in 
nature  (summarized  in  the  middle  row),  where  various  actors  agree  to  support  legal  rights  for  all  as 
a  means  of  securing  their  own  (1996). 

Groups  based  on  solidarity  (and  its  diverse  bases)  may  draw  upon  any  of  the  protogame  strategies, 
depending  on  their  historic  and  current  relationships.  Groups  can  view  each  other  as  allies  (with 
whom  they  have  an  affinity),  competitors  and/or  actors  with  asynchronous  dependencies  (cf. 
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Emerson,  1962)  (with  whom  they  engage  in  a  pragmatic  or  instrumental  way),  or  enemies  (whom 
they  view  as  having  to  be  coerced,  or  be  threatened  with  coercion).  In  each  case,  the  calibration  of 
particular  moves  depends  upon  the  specific  situation,  and  on  the  actors'  understanding  of  it. 


3.  Modeling  Gaps 

Research  has  made  considerable  progress  in  modeling  a  wide  variety  of  social  systems,  ranging 
from  individual  cognition  to  organizational  competition.  Nevertheless,  these  models  are  not 
perfect.  Below  we  note  several  gaps  for  each  of  the  models  introduced  in  this  chapter. 

Blau  space  models  offer  data-efficient  approaches  to  modeling  abstract  dynamics  at  a  high  level, 
but  applying  them  involves  more  art  than  science.  No  rigorous  method  exists  for  determining 
which  variables  to  select  as  Blau  dimensions.  The  only  theoretical  guideline  is  that  the  dimensions 
be  "salient"  to  the  population  under  study,  meaning  that  the  variables  must  have  an  impact  on 
association  (Blau,  1977).  In  general,  any  variable  on  which  association  is  homophilous  is  a  salient 
dimension,  but  this  is  a  weak  criterion.  Further,  the  modeling  approach  has  no  guidelines  for  the 
number  of  dimensions  to  include.  Larger  numbers  of  dimensions  tend  to  reduce  the  overlap 
between  niches  while  smaller  numbers  increase  it,  but  no  method  exists  for  determining  an 
appropriate  number.  This  approach  would  therefore  benefit  from  methods  to  rapidly  and  reliably 
assess  whether  a  given  Blau  space  model  has  the  correct  dimensions. 

The  types  of  variables  that  can  be  used  as  dimensions  also  depend  on  whether  the  analysis  focuses 
on  individuals  or  organizations.  When  depicting  organizations,  the  models  can  include  categorical 
variables  (e.g.,  race,  sex)  by  rendering  a  dimension  as  the  percentage  of  a  group  that  falls  into  a 
particular  category  (e.g.,  percentage  of  the  organization  that  is  female).  However,  this  approach 
does  not  work  when  modeling  individuals  because  they  have  discrete  values  (e.g.,  male  or  female); 
as  a  result,  a  Blau  space  model  of  individuals  can  use  only  continuous  variables  (e.g.,  income,  age) 
as  dimensions.  New  methods  of  scaling  Blau  space  dimensions  are  needed  to  eliminate  this 
restriction. 

Social  network  models  have  made  tremendous  strides  in  recent  decades,  but  require  substantial 
improvement.  First,  ERGMs  parameterize  specific  network  configurations,  but  highly  preferred 
configurations  sometimes  lack  a  clear  meaning.  In  other  words,  a  particular  configuration  may  be 
necessary  to  achieve  acceptable  model  fit,  but  there  may  be  no  clear  process  through  which  this 
configuration  could  emerge.  Research  should  determine  whether  these  odd  configurations  signal  a 
deep  insight  about  social  behavior  that  remains  to  be  achieved  or  simply  indicate  that  the  model  is 
degenerate. 

Additionally,  while  ERGMs  and  Siena  models  are  powerful  techniques,  they  both  rely  on  simulation. 
This  requires  a  considerable  amount  of  computation  time,  which  can  be  prohibitive  in  large  and 
complex  networks.  Developers  should  devote  additional  work  to  optimizing  these  techniques  for 
rapid  computation,  preferably  by  exploiting  multi-core  and  parallel  computing  architectures 
whenever  possible. 
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SINT  delivers  a  powerful  and  flexible  model  of  attitude  change  and  stability,  but  several  hurdles 
remain.  First,  can  SINT  be  adapted  also  to  model  behaviors?  Individuals  may  be  more  likely  to 
adopt  a  particular  behavior  or  practice  as  their  attitude  toward  it  improves,  but  no  link  function 
that  relates  attitudes  to  behaviors  currently  exists.  It  should  be  possible  to  specify  such  a  link,  but 
doing  so  would  impose  assumptions  about  the  causal  linkage  between  attitudes  and  behaviors. 
Given  the  pronounced  issues  of  reverse  causation  in  this  area  (i.e.,  attitudes  may  follow  from 
behaviors),  developing  such  a  link  function  is  a  theoretical  and  empirical  task  as  much  as  a 
mathematical  one.  Second,  including  an  actor's  attitude  at  the  first  time  point  in  every  iteration 
prevents  perfect  consensus  from  emerging  in  all  cases,  but  also  gives  one  arbitrary  moment  a 
disproportionate  influence  over  the  model.  SINT  would  benefit  from  development  of  an  alternative 
approach  to  preventing  perfect  consensus  from  emerging  in  all  cases. 

The  majority  of  the  models  reviewed  in  this  chapter  were  originally  developed  with  face-to-face 
interactions  in  mind,  but  interactions  increasingly  occur  in  electronically  mediated  forms.  It  would 
be  useful  to  take  advantage  of  the  wealth  of  online  data  on  social  behavior,  but  models  developed 
around  offline  interactions  may  not  translate  to  online  contexts.  Some  models  (e.g.,  SINT)  have 
enough  flexibility  to  handle  the  transition  with  minimal  modification,  but  other  models  are  likely  to 
be  context  dependent.  This  necessarily  means  that  we  must  exercise  restraint  in  using  online  data 
until  our  understanding  has  improved,  but  completing  this  basic  research  will  produce  substantial 
long-term  advantages,  as  electronically  mediated  communication  can  only  continue  to  grow  in 
importance. 

The  integration  of  cognitive  and  game-theoretic  models  raises  four  essential  issues:  1)  How  can  a 
broad  range  of  social  theories  be  integrated  with  each  other  and  incorporated  into  computational 
models  of  social  dynamics?  2)  How  can  social  models  best  handle  cross-scale  interaction?  3)  How 
can  endogenous  models  of  situated  understanding  best  capture  the  rich  interactions  among  social 
actors  and  still  remain  computationally  tractable?  4)  What  are  the  mathematical  foundations  for 
effective  models  of  social  dynamics?  These  questions  define  further  gaps  for  research  to  address. 

The  first  and  last  questions  are  fundamental.  Answering  them  will  provide  the  ultimate  basis  for 
the  advancement  of  computational  models.  The  second  and  third  questions  are  more  application 
oriented.  They  provide  a  focus  to  the  more  abstract  questions,  as  well  as  the  criteria  for 
prospective  answers.  Research  addressing  all  of  these  questions  will  result  in  a  new  generation  of 
computational  architecture,  and  a  new  foundation  for  computational  social  science. 

3.1.  The  Road  to  Transition 

What  steps  must  developers  take  to  convert  the  models  discussed  here  into  operational  systems 
ready  for  use  by  practitioners?  One  of  the  most  extensive,  and  necessary,  improvements  involves 
creating  robust  systems  for  converting  between  data  formats.  The  models  described  above  are 
implemented  in  a  number  of  different  programming  languages,  and  one  of  the  greatest  routine 
difficulties  in  using  them  stems  from  the  need  to  transform  data  to  be  compatible  with  each.  If  and 
when  translation  protocols  emerge,  developers  must  exercise  care  in  minimizing  the  computational 
time  required  to  store  and  transform  the  data. 
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Each  of  the  described  models  has  achieved  significant  empirical  success,  but,  at  present,  guidelines 
for  identifying  degenerate  or  poorly  fitting  models  remain  loose.  For  practitioners  to  apply  these 
models  effectively,  they  must  be  able  to  determine  clearly  whether  the  models  are  fitting 
adequately  and  to  diagnose  the  inevitable  problems  that  arise  in  any  analytic  task.  Therefore, 
developers  must  also  create  and  automate  diagnostics  for  the  models  described  in  this  chapter. 

Despite  their  separate  developmental  histories,  many  of  the  models  presented  in  this  chapter  have 
deep  similarities.  For  example,  ERGMs  work  on  networks  assumed  to  be  in  equilibrium,  whereas 
Siena  models  focus  on  disequilibrium.  Similarly,  both  SINT  and  Siena  focus  primarily  on  the 
transition  to  stability,  although  SINT  includes  explicit  equations  for  equilibrium  cases.  These 
connections  suggest  the  possibility  of  integrating  these  models  into  a  general  framework  useful  in  a 
variety  of  contexts.  Successful  integration  would  reduce  the  training  time  necessary  for 
practitioners  to  perform  many  of  these  types  of  analyses  competently. 

This  discussion  of  gaps  defines  a  framework  for  progress  in  cognitive  modeling  and  game  theory. 
Beyond  that  framework,  the  transitional  path  concerns  the  type  of  computational  architecture  in 
which  these  questions  cannot  only  be  addressed,  but  can  also  evolve  as  innovations  give  rise  to 
new  issues.  The  broad  computational  form  will  probably  be  based  on  agent-based  modeling  and 
simulation,  a  topic  deemed  beyond  the  scope  of  the  present  chapter.  However,  developers  should 
view  the  present  forms  of  agent  simulation  as  bare  bones  relative  to  the  capabilities  ultimately 
required.  The  framework  defining  the  gaps  also  identifies  the  transitional  path.  We  can  expect  that 
agent  simulation  in  the  future  will  be  far  more  theoretical  and  extensively  grounded  in 
mathematical  relations  and  transformations  (cf.  Sallach,  2012).  As  game  formality  increases,  it  will 
not  only  specify  simulations,  but  also  form  a  basis  for  deductive  analysis.  Each  of  these  advances 
will  transform  and  extend  computational  social  science,  allowing  it  to  realize  its  full  potential. 


4.  Conclusions 

Research  has  made  substantial  progress  in  developing  and  refining  computational  models  of 
human  behavior,  and  has  produced  a  palette  of  models  useful  for  a  broad  range  of  specific 
contexts  and  data  limitations.  Additional  work  is  needed— both  basic  research  to  improve  our 
understanding  of  the  workings  and  implications  of  the  models  and  applied  research  to  make  them 
easier  for  practitioners  to  use— but  the  basic  groundwork  has  been  completed. 

The  models  discussed  in  this  chapter  aid  in  understanding  the  operation  of  social  groups  in  a 
variety  or  respects.  They  can  provide  clues  as  to  the  processes  that  yield  specific  network 
configurations  and  the  likelihood  of  different  configurations,  can  describe  the  degree  of 
competition  between  organizations  and  other  abstract  entities,  and  can  explain  the  inner  workings 
of  attitude  change.  We  must  reemphasize  that  each  model  described  here  is  useful  in  particular 
circumstances  and  depends  on  specific  assumptions.  Practitioners  should  therefore  assess  the 
usefulness  of  a  model  on  the  basis  of  these  assumptions  and  of  their  validity  in  a  particular 
modeling  context.  Assumptions  will  often  be  strained,  if  not  actually  violated,  but  practitioners 
should  take  care  to  ensure  that  these  violations  remain  within  acceptable  bounds.  This  chapter  has 
surveyed  many  models,  and  made  no  attempt  to  describe  the  various  assumptions  made 
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exhaustively,  but  interested  readers  can  consult  the  literature  cited  to  learn  more  about  these 
specific  techniques  and  their  advantages  and  disadvantages. 

Computational  models  lend  themselves  to  diverse  theories  and  their  integration.  Even  as 
researchers  assess  the  theories  that  animate  them  more  broadly  and  theoretically,  the  models 
must  retain  their  expressiveness.  Drawing  on  the  insights  attained,  their  formulation  as 
mathematical  models,  more  elegant  and  more  compelling,  will  contribute  greatly  to  the  advance  of 
mathematical  sociology.  In  20  years,  we  can  expect  that  social  models  will  have  become 
considerably  more  effective  because  of  the  mutual  strengthening  of  theoretical  sociology  and  the 
mathematical  formulations  of  these  theories. 
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1.  Introduction 

Visualization  for  sociocultural  understanding  involves  the  data,  tools,  and  techniques  necessary  to 
convey  the  behavioral  relationships  underlying  daily  life,  relationships  expressing  the  evolution  of  a 
culture,  and  the  social  structures  in  which  these  relationships  occur.  Creating  visualizations  to 
better  understand  sociocultural  behavior  helps  decision  makers  and  end-users  build  situational 
awareness  and  effective  operational  engagement. 

Because  sociocultural  behaviors  are  multidimensional,  historically  contextualized,  and  emblematic 
of  the  ontological  structure  of  the  society,  visualizing  the  understanding  of  that  sociocultural 
context  indeed  presents  a  daunting  task.  While  no  accepted  set  of  protocols  exists  to  measure 
success  in  such  an  undertaking,  specific  aspects  of  individual  and  collective  engagement  can  be 
deemed  successful  if  those  engagements  lead  to  an  appreciation  of  the  other's  perspective. 

This  chapter  highlights  some  of  the  advances  in  virtualization  technology,  social  interactive 
modeling,  and  perceptual  science  that  have  opened  vast  frontiers  in  visualization,  especially  in 
multidimensional  space.  Visualization  as  a  theoretical  construct  has  precedence  in  Card,  Mackinley, 
and  Shneiderman  (1999),  which  provides  an  exhaustive  study  of  research  on  multidimensional 
visualization,  while  Tufte  (1997,  2001)  has  written  extensively  on  graphical  representation  of 
quantitative  and  qualitative  data  as  part  of  information  design.  In  this  chapter,  we  build  upon  these 
earlier  works  to  explore  approaches  to  visualizing  data  useful  in  furthering  understanding  of  the 
sociocultural  environment. 

Visualizing  a  society's  sociocultural  structure  has  two  primary  benefits.  Human  interactions  and 
relationships  provide  the  context  for  social  inquiry.  By  visualizing  these  interactions  and 
relationships,  the  researcher  can  better  identify  the  most  effective  means  for  establishing  a 
presence,  assembling  and  tailoring  the  necessary  tools  for  engagement,  and  determining  desired 
outcomes  and  measures  of  success.  Secondly,  constraining  the  complexity  of  social  engagement 
through  visualization  brings  the  key  elements  of  the  potential  engagement  into  focus  and  provides 
the  roadmap  by  which  intervention  can  occur. 

Visual  representation  of  complex  human  interactions  leads  to  a  more  informed  understanding  of 
the  defining  characteristics  of  the  society  and  the  nuances  of  its  cultural  practices.  The  diversity  of 
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social  interactions,  however— whether  traditional  dances  celebrating  the  harvest,  the  prolific  use 
of  hand  gestures  in  public  conversations,  or  the  ever-increasing  acceptance  of  social  media  as  a 
proxy  for  presence— makes  isolating  attributes  that  uniquely  characterize  a  given  society  an 
unrealistic  goal.  Sociocultural  practices  are  fluid  and  malleable  by  the  external  influences  that  sway 
practices  and  behaviors.  Practices,  attitudes,  and  behaviors  once  considered  acceptable,  whether 
denying  human  rights  to  select  groups  or  according  divine  attributes  to  others,  have  undergone 
alterations,  adaptations,  or  outright  rejection  as  societies  evolve  (or  devolve). 

Capturing  what  makes  a  society  unique  is  further  complicated  by  the  many  different  theoretical 
frameworks  in  which  visualization  of  such  understanding  occurs.  Purchase,  Andrienko,  Jankun-Kelly, 
&  Ward  (2008)  posit  three  distinct  conceptual  mechanisms  in  visualizing  understanding: 

1.  Visualization  as  an  interpreted  activity  representing  the  physical  form; 

2.  Exploration  and  manipulation  of  the  external  representation  to  understand  an  underlying 
model  of  the  external  form;  and 

3.  Investigation  of  the  posited  model  to  discover  interrelationships,  trends,  and  patterns. 

These  three  frameworks  are  not  categorically  distinct.  Visualizing  understanding  is  a  dynamic 
process,  and  fully  exploring  the  social  interactions  in  a  culture  requires  elements  of  all  three 
frameworks.  From  a  sociological  perspective,  visualization  draws  upon  data-centric  interactions 
between  an  observer  and  her  participant,  which  can  be  organized  into  a  predictive  capability  by 
coupling  data  analysis  with  a  modeling  approach  that  is  transferrable  across  cultural  systems  and 
identifies  underlying  dynamics  of  symbolic  and  economic  exchange. 

Many  different  types  of  data  are  available  to  provide  a  relative  snapshot  of  a  society's 
organizational  structure.  These  data  are  ubiquitous  resources  in  traditional  sociological 
investigative  methods  and  often  provide  historical  context  to  facilitate  the  identification  of  trends 
within  the  society  as  its  institutions  mature.  Examples  of  these  data  include  demographic  data, 
economic  data  characterizing  household  compositions  and  major  industrial  drivers  of  a  society, 
environmental  data  encompassing  quality-of-life  factors,  and  sociological  surveys  that  give  insight 
into  the  population's  overall  assessment  of  their  society  and  the  individual's  place  within  that 
society.  In  addition  to  these  traditional  methods  of  understanding  the  sociocultural  variability  of  a 
people  are  the  relatively  new  tools  offered  by  social  media  networks  and  the  vast  potential  of 
visualizations  afforded  by  virtual  interactions. 

For  many  marginalized  populations,  traditional  datasets  are  at  best  sparse  and  of  poor  quality,  and 
more  often  nonexistent.  The  kinds  of  data  and  data  collection  methods  made  available  by  social 
media  supplement  or  even  bypass  traditional  data  collection  methodologies,  generating  an  array  of 
new  datasets  that,  when  visualized,  provide  fresh  sociocultural  insights.  Social  media  formats  such 
as  Facebook,  Twitter,  and  Flickr  as  examples,  offer  researchers  new  tools  to  connect  with,  engage 
in,  and  understand  an  array  of  interests  and  current  events  in  near-real  time.  These  new  media  also 
create  new  challenges,  as  the  ever-expanding  stream  of  data  magnifies  the  traditional  difficulties  of 
understanding  the  cultural  and  linguistic  signatures  of  a  society  embedded  in  those  data.  To  reap 
the  advantages  of  the  new  data  streams,  researchers  must  now  develop  tools  for  efficient  ingest  of 
large  amounts  of  social  media  data,  tools  and  theory-based  methods  for  interpreting  the  data,  and 
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tools  for  effective  visualization  of  vast  datasets,  while  simultaneously  protecting  the  privacy  of 
individual  subjects. 

The  chapter  begins  with  a  discussion  of  the  various  data  types  available  for  sociocultural 
visualization  of  understanding.  Section  2  elaborates  upon  the  different  data  types  used  to 
encapsulate  our  contextual  understanding  of  social  actors.  Section  3  presents  an  example  analysis 
of  a  linguistic  network  in  which  group  identification  is  transformed  into  recognizable  patterns 
yielding  actionable  intelligence  to  guide  subsequent  engagements.  Section  4  introduces  traditional 
data  representation  methods  geared  to  quantifying  the  social  environment,  whether  demographic 
analysis  or  economic  snapshots  of  individuals,  groups,  or  countries.  Section  5  explores  data 
visualization  in  a  multi-dimensional  space  and  the  tools  necessary  to  represent  that  space.  Section 
6  provides  an  overview  of  the  new  streaming  methods  of  data  amalgamation  and  visualization, 
while  section  7  completes  our  discussion  of  the  challenges  created  by  this  streaming  data: 
unconstrained  visualization  without  a  priori  hypothetical  structure. 


2.  Data  Collection  Methods 

Understanding  a  particular  sociocultural  environment  requires  data.  Regardless  of  the  method 
employed,  data  collection  defines  the  key  elements  of  concern  for  researchers  who  hope  to  garner 
insights  about  a  given  social  system.  It  also  parameterizes  how  the  researcher  can  visualize  the 
extracted  data  to  test  various  hypotheses  about  the  social  organization  under  examination.  As  with 
any  scientific  investigation,  researchers  should  understand  the  visualization  requirements  in  order 
to  specify  the  data  to  collect,  the  duration  of  the  data  collection,  and  the  means  for  visualizing  the 
data  amassed. 

Given  the  amount  of  data  accessible  for  visualization  we  must  first  define  the  various  data 
elements  of  interest.  Data  collection  methodologies  organize  data  into  the  formats  in  which  they 
will  be  stored,  processed,  and  ultimately  visualized.  These  formats  fall  into  three  categories: 
structured  data,  semi-structured  data,  and  unstructured  data. 

2.1  Structured  Data 

Structured  data  formats  store  data  elements  in  a  systematic  organization;  for  example,  within  a 
database  management  system.  These  data  are  often  gleaned  from  and  stored  in  data  tables  with 
constraints  placed  upon  the  storage  requirements,  whether  in  terms  of  specific  data  types  (date, 
number,  character,  blob)  or  of  specific  data  categorization  (nominal,  ordinal,  integer,  ratio). 
Structured  data  are  easy  to  manipulate  using  a  structured  query  language  and  are  amenable  to  a 
multiplicity  of  data  visualizations  (e.g.,  charts,  graphs,  maps,  tables). 

Table  1  shows  an  example  of  a  data  collection  from  the  Tropical  Atmosphere  and  Ocean  (TAO) 
buoy  array  (McPhaden  et  al.,  1998).  Seventy  moored  buoys  are  strategically  deployed  in  the 
equatorial  Pacific  Ocean  to  capture  a  multiplicity  of  climatic  data  that  parameterize  the 
atmosphere  -  ocean  interface.  Sensors  on  each  buoy  collect  data  at  defined  intervals  and  transmit 
those  measurements  to  the  ARGOS  satellite  relay  system.  The  data  are  then  processed  and 
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assimilated  into  climate  models  to  better  visualize  the  dynamics  of  the  Southern  Oscillation  and  its 
perturbations,  El  Nino  and  La  Nina. 


Table  1.  Data  collection  metrics  for  structured  data  elements 


Location:  ON  156E  9  Feb  1993  to  20  Mar  2004  (  4058  times,  7  blocks)  Gen.  Date  Mar  21  2004 
Units:  Winds  (m/s),  W.  Dir  (deg),  AirT  (C),  SST  (C),  Rel.  Flumidity  (%),  -99.9  =  missing.  Winds  Use 
Oceanographic  Conventional, 1)  is  NE  at  sqrt(2)  m/s.  Time:  1200  9  Feb  1993  to  1200  12  Dec  1994 
(index  1  to  672,  672  times) 

Depth  (M): 

-4 

-4 

-4 

-4 

-3 

1 

-3 

QUALITY 

YYYYMMDD 

HHMM 

UWND 

VWND 

WSPD 

WDIR 

AIRT 

SST 

RH 

S  DAT  FI 

19930209 

1200 

2.5 

-1.0 

2.7 

112.3 

27.70 

29.25 

81.7 

22222 

19930210 

1200 

4.2 

-0.5 

4.2 

9.2 

26.98 

29.17 

86.1 

22222 

19930211 

1200 

0.4 

-3.0 

3.0 

171.8 

27.21 

29.07 

87.1 

22222 

19930212 

1200 

-1.1 

-3.0 

3.2 

199.6 

28.11 

29.15 

82.2 

22222 

Note :  Data  from  McPhaden  et  al.  (1998) 


The  key  to  understanding  a  structured  data  collection  plan  lies  in  the  measurability  of  each  element 
collected.  Measurable  attributes  of  the  data  collection  process  include  the  duration  of  data 
collection,  the  sample's  spatial  domain  (if  applicable),  and  the  start  and  end  dates  for  data 
collection.  Additional  data  may  be  presented  on  data  quality  as  a  means  to  guide  the  usability  of 
the  selected  dataset.  Data  quality  attribution  enables  researchers  to  assess  the  overall  value  of  the 
data  collection  process. 

2.2  Semi-Structured  Data 

Semi-structured  formats  do  not  normalize  the  data  into  clearly  defined  tables,  but  rather  group 
them  by  semantic  similarities  in  format  and  content.  Data  presented  in  Extensible  Markup 
Language  (XML)  format  or  the  geographic  data  type  Keyhole  Markup  Language  (KML)  format  are 
examples  of  semi-structured  data.  Figure  1  provides  an  example  of  an  XML  snippet  showing  the 
metadata  for  a  given  traffic  alert. 
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<?xml  version="1.0"  encoding="UTF-8"?> 

<ResultSet  xmlns:xsi- 'http://www.w3.org/2001/XM LSchema-instance" 
xmlns="urn:yahoo:maps"  xsi:schemaLocation="urn:yahoo:maps 
http://api.local.yahoo.com/MapsService/Vl/TrafficDataResponse.xsd"><LastUpdat 
eDate>1262722386</LastUpdateDate><Result  type="incident"xTitle>Closed,  on 
RT-440  SB  at  COMMUNIPAW  AVE</TitlexDescription>ROAD  CLOSED;  DUE  TO 
WATERMAIN  BREAK  REPAIRS  FOLLOW  POLICE  DETOURS  ALSO  THE  RAMP  TO 
SOUTHBOUND  440  FROM  COMMUNIPAW  AVENUE  IS  CLOSED  AS  WELL  AS 
NORTHBOUND  TRUCK  RT  19  TO  RT  440  NOTE:  DOT  REPORTS  THIS  WILL  REMAINS 
CLOSED  UNTIL  FURTHER  NOTICE.  SEEK 

ALTERNATE.</DescriptionxLatitude>40.723743</LatitudexLongitude>- 

74.090166</LongitudexDirection>SB</DirectionxSeverity>4</SeverityxReportD 

ate>1262599901</ReportDatexUpdateDate>1262699951</UpdateDatexEndDate 


Figure  1.  Traffic  alert  using  Really  Simple  Synchronization  (RSS)  messaging. 


The  above  output  is  a  semi-structured  text  message  that  the  data  collector  would  transfer  to  a 
mobile  device  to  alert  drivers  to  a  water  main  break  in  the  specified  area.  These  data  are 
embedded  within  a  metadata  processing  language  that  is  human  readable  but  can  only  be  fully 
visualized  through  a  human-computer  interface  such  as  a  Simple  Message  Service  (SMS)  reader. 
Figure  2  shows  the  visualization  of  this  message  as  a  GeoRSS  feed  using  the  Bing  Map  interface. 
Note  how  easily  a  reader  can  understand  the  impacted  area  thanks  to  the  additional  spatial  context 
that  results  from  showing  the  road  network  with  easily  identifiable  roads  and  traffic  avoidance 
areas.  While  the  SMS  translates  the  XML  tags  into  a  readable  text  message,  the  GeoRSS  provides 
another  dimension  of  detail  in  visualizing  the  output,  placing  the  messaging  service  into  a 
geographic  context  complete  with  a  road  network  and  geopolitical  boundaries  that  identify  the 
area  in  which  the  water  main  break  occurred. 
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Figure  2.  Above  traffic  alert  visualized  using  GeoRSS  in  the  Bing  map  interface.  Adapted  from  Bing 
(2013). 


2.3.  Unstructured  Data 

Unstructured  data  encompasses  the  realm  of  data  objects  represented  by  the  semi-natural, 
irregular  language  of  social  media,  as  represented  in  blogs,  instant  messaging,  Facebook  posts, 
Instagrams,  tweets,  among  others.  To  leverage  such  data  for  sociocultural  understanding, 
researchers  need  to  capture  the  data  and  integrate  them  into  an  analytical  methodology,  which  is 
antithetical  to  the  very  premise  of  unstructured  data  as  a  free-form  collection  that  resists  a  priori 
hypotheses  regarding  its  internal  structure.  This  is  the  very  challenge  that  now  defines  the  realm  of 
Big  Data  and  its  analysis.  In  essence,  the  difficulty  centers  on  detecting  the  signal  embedded  within 
the  noise,  differentiating  patterns  of  activities  from  outliers,  and  determining  with  some  certainty 
that  the  patterns  generated  carry  sociocultural  meaning.  Figure  3  gives  a  snapshot  of  unstructured 
data  in  the  form  of  a  Twitter  feed  from  the  Kenyan  Ministry  for  the  State  of  Defense  during  Kenya's 
war  on  the  Al-Shabaab  terrorist  network  in  November  2011. 
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@MajorEChirchir 


#operationlindanchi  Kenyans  dealing  in 
donkey  trade  along  the  Kenya-Somali 
border  are  advised  not  to  sell  their 
animals  to  A1  Shabaab 


Favorite  xx  Relweet  ^  Reply 


Figure  3.  Tweet  advising  Kenyans  not  to  sell  animals  to  Al-Shabaab.  Adapted  from  Twitter  (2011), 
Maj.  Chirchir  Twitter  feed. 


This  same  Twitter  account,  however,  posted  manipulated  images  of  an  individual  allegedly  stoned 
by  Al  Shabaab  militants  in  order  to  provoke  nationalistic  outrage  within  Kenya.  Sentiment 
manipulation  illustrates  why  researchers  should  approach  these  new  media  technologies  with 
caution 

Visualization  of  unstructured  data  requires  new  techniques  to  identify  the  relative  importance  of 
the  various  data  elements.  These  data  are  not  necessarily  accessible  to  the  traditional  metrics  of 
structured  data.  Figure  4  shows  an  example  of  visualizing  the  predominance  of  registered  users 
among  various  social  networking  sites.  In  this  visualization,  font  sizes  scale  to  relative  numbers  of 
registered  users.  To  read  this  graphic,  consider  the  example  of  "facebook."  This  social  media 
platform  garners  far  greater  numbers  of  registered  users  relative  to  the  other  media  providers.  This 
distinction  is  evident  by  the  larger  font  size.  In  contrast,  Studivz,  a  social  media  platform  geared  to 
university  students  in  Central  Europe,  is  referenced  by  a  much  smaller  font  size,  indicating  a  smaller 
user  base. 


51.com  baidu  bebo  classmates.com  cyworld 


TaceDOOK  friendster  hdbbo  hi5  k 

myspace  myyearbook  nasza-klasa.pl  netlog  odnoklassniki  Orkut 


kaixin  mixi 


renren  skyrock  sonico.com  studivz  tagged  vkontakte  xanga 


Figure  4.  Relative  preponderance  of  social  media  platforms,  November  2012. 
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3.0.  Organizing  the  Conceptual  Framework  for  Visualizing  Understanding 

As  outlined  in  the  introduction,  visualization  can  be  conceptualized  using  three  different, 
overlapping  paradigms.  Visualization  of  understanding  as  an  interpretation  of  external  form,  as  a 
discovery  tool,  and  as  an  exploration  of  the  internal  cohesiveness  of  the  visualized  patterns  lead 
researchers  to  use  different  data  collection  methods  and  formats  to  parameterize  sociocultural 
interactions.  A  meta-discussion  of  the  models  and  tools  can  increase  understanding  of  inherent 
bias  in  the  methodology  employed  to  comprehend  a  particular  social  group  or  cultural  practice. 
Clear  benefits  accrue  from  using  visualizations  appropriate  to  the  particularities  of  the  data 
scrutinized.  We  next  describe  some  of  the  practicalities  of  visualizing  sociocultural  data  through 
these  frameworks  and  their  use  in  representing  the  underlying  sociocultural  relationships.  We 
examine  the  conceptual  frameworks  within  the  context  of  (a)  linguistic  networks  of  convergent 
ideas  as  manifested  in  group  rhetoric,  (b)  affinity  diagrams  showing  the  organization  of 
unstructured  data,  (c)  conceptual  mapping,  and  (d)  link  analysis  of  known  social  networks. 

3.1.  Linguistic  Networks 

Michael  Gabbay  and  Ashley  Thirkill-Mackelprang  (2011)  developed  a  deterministic  framework  for 
characterizing  the  various  insurgent  groups  during  the  Iraq  war  (2003-2009).  Their  research 
distinguished  among  groups  based  upon  the  rhetoric  the  groups  used  and  defined  two  broad 
categories  of  insurgents:  the  Jihadist  Salafists  and  Nationalists.  The  Jihadist  Salafists  (for  instance, 
Al  Qaida  in  Iraq  and  Ansar  al-lslam)  adhere  to  a  strict,  orthodox  interpretation  of  Islam  and  identify 
themselves  with  a  pan-lslamic  caliphate.  The  authors  characterized  these  groups  as  unequivocally 
hostile  to  Shiites  and  to  groups  affiliated  with  the  Iraqi  government.  In  contrast,  Nationalist  groups 
in  Iraq  emphasize  the  geographic  and  ethnographic  integrity  of  all  Iraqi  citizens  and  favor  a  central 
government  espousing  a  basis  in  Islamic  law.  Nationalist  groups  express  varying  degrees  of  anti- 
Shiite  sentiment,  but  overall  their  key  rhetoric  centered  on  maintaining  a  united  Iraq  for  Shiites  as 
well  as  other  ethnic  groups  and  sects  within  the  country. 

Gabbay  and  Thirkill-Mackelprang  posit  that  rhetoric  constitutes  the  central  political  observable  of 
an  insurgency  because  it  delineates  the  degree  to  which  an  insurgent  group  tolerates  and  /  or 
celebrates  violence  to  meet  its  political  objectives.  The  authors  devised  a  series  of  "conflict  frames" 
to  leverage  this  unstructured  data  and  identify  in-groups  (i.e.,  insurgent  groups  with  similar 
political  objectives  and  willingness  to  engage  in  violent  acts)  and  out-groups  (i.e.,  the  U.S.  military, 
its  allies,  Iraqi  National  Security  Forces,  and  local  groups  willing  to  collaborate  with  U.S.  forces).  The 
unstructured  data  used  in  this  study  encompassed  over  2,000  translated  statements  from  18  of  the 
most  vocal  insurgent  groups.  Table  2  provides  an  example  of  the  kinds  of  in-  and  out-groups  that 
Gabbay  and  Thirkill-Mackelprang  identified  throughout  the  Iraq  war  as  well  as  the  various  marker 
terms  used  to  structure  the  degree  of  rhetorical  violence  with  which  these  groups  propagandized 
their  respective  causes. 
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Table  2.  Rhetorical  marker  terms  and  in-  and  out-group  assignment. 


Marker  Term 

In-Groups 

Out-Groups 

agent  government 

Iraqi  Government 

apostate  government 

Sunni  Civil  Society 

Iraqi  Government 

companions  of  the  prophet 

Sunni  Civil  Society 

cross  worshippers 

Sunni  Civil  Society 

United  States 

Iraqis 

Iraqi  Civil  Society 

Iranian  occupations 

Sunni  Civil  Society 

Iraqi  Government,  Shiite  Political  Parties 

Occupier 

United  States 

Rejectionist 

Sunni  Civil  Society 

Shiite  Civil  Society 

Rescue  council 

Sunni  militias  (Awakening  Councils) 

Note:  Adapted  with  permission  from  Gabbay  &  Thirkill-Mackelprang  (2011) 


3.2.  Context  Inquiry  and  Affinity  Diagrams 

Context  inquiry  offers  an  excellent  way  to  explore  group  dynamics  and  the  individual's  role  within  a 
group.  In  this  approach  the  researcher  meets  with  a  subject  for  scheduled  observational  interviews 
during  which  time  the  subject  continues  his  or  her  daily  routines  and  interactions  with  other 
members  of  the  community.  Through  observation  and  interviews,  the  researcher  begins  to 
understand  and  interpret  the  subject's  actions  as  informed  by  and  consequent  to  the  larger  group 
dynamics.  Context  inquiry  offers  the  tactical  advantage  of  affording  subjects  an  opportunity  to 
develop  a  narrative  about  their  personal  lives  and  the  interrelationships  informing  the  larger  social 
context. 

This  kind  of  research  produces  structured  or  unstructured  data,  depending  on  whether  the 
researcher  collects  measurable  attributes  (e.g.,  number  of  calories  consumed  per  day,  hours  of 
sleep  per  night)  or  information  derived  from  conversations  (e.g.,  survey  instruments,  informal 
conversations).  In  either  case,  the  framework  for  visualizing  the  interactions  can  take  the  form  of  a 
simple  graph  of  daily  calorie  intake  or  hours  performing  specific  tasks,  a  link  chart  of  the  social 
network  and  communications  relay,  or  a  detailed  model  of  the  subject's  daily  life  as  a  micro 
expression  of  the  larger  social  interactions  defining  the  social  and  cultural  norms.  This  type  of 
inquiry  tool  is  also  amenable  to  scrutinizing  the  kinds  of  preconceived  notions  that  informed  the 
observer's  understanding  of  the  society  prior  to  this  intimate  view  at  the  personal  level. 
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Researchers  can  use  affinity  diagrams  to  structure  the  visualization  of  data  obtained  from  a  context 
inquiry.  To  generate  an  affinity  diagram,  the  researcher  sorts  various  data  snippets,  whether 
expressions  used  to  characterize  an  interaction,  the  flow  of  an  interviewee's  daily  tasks,  the 
hierarchical  structure  of  civil  institutions,  etc.,  into  groups  based  on  their  natural  relationships. 
These  groups  may  be  further  sorted  into  subgroups  to  better  facilitate  a  logically  coherent  analysis 
of  daily  observations.  The  affinity  diagram  graphically  conveys  groupings  and  relationships.  Figure  5 
shows  an  example  of  a  disordered  set  of  observations  structured  into  a  hierarchical  diagram  that 
organizes  thoughts  and  ideas  for  better  understanding.  Affinity  diagrams  are  designed  as 
brainstorming  tools  and  as  a  free  form  collection  of  information  snippets  that  can  be  easily  moved 
from  one  category  to  another  as  new  information  is  posted  to  the  diagram  or  as  the  researcher 
reorganizes  categories  of  the  hierarchy. 


Brainstorming  Angels 


Do  angels  exist  in 
eternity? 


Do  angels  have  bodies? 


Can  angels  be  in  several 
places  at  the  same  time? 

Do  angels  know  of  the  future? 

Can  angels  err? 


Do  angels  possess  self 
knowledge? 


Do  angels  occupy  space? 


Do  angels  exercise 
bodily  functions? 


Are  angels  aware  of  each 
other? 


Do  angels  possess  self- 
know  letfee? 


Are  angels  composed  of 
matter  and  form? 


Do  angels  move 
through  time? 


Do  angels  possess  free 

will? 


Figure  5.  Affinity  diagram  structuring  data  elements  into  logical  categories. 


Context  inquiry  also  affords  the  opportunity  for  subjects  to  self-reflect  on  previously  unrecognized 
aspects  of  their  daily  lives,  based  upon  interpretations  offered  by  the  researcher.  Because  affinity 
diagrams  are  designed  to  facilitate  discussion,  the  researcher  can  gauge  the  accuracy  of 
interpretations  by  the  degree  of  validation  offered  by  the  subject  to  the  derived  categories  or  even 
the  rationale  used  to  organize  data  into  a  given  category. 
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3.3.  Concept  Maps 

Concept  maps  are  two-dimensional  graphic  visualizations  of  knowledge  concepts  and  the 
propositions  that  actively  join  concepts  in  one-to-one,  one-to-many,  or  many-to-many  entity 
relationships.  They  offer  the  flexibility  of  multiple  relations  occurring  across  one  or  many  domains, 
which  makes  them  ideal  visualization  tools.  As  framework  tools  for  visualizing  understanding,  these 
maps  facilitate  an  understanding  of  the  external  organization  in  which  individuals  interact,  can  help 
researchers  to  identify  power  relations  within  an  organized  structure  such  as  a  government  entity 
or  an  institution,  and  can  be  used  to  comprehend  the  internal  workings  of  a  group  acting  within  the 
larger  social  organization.  Concept  maps  sketch  the  entities  defining  a  domain  of  knowledge  in 
such  a  manner  that  the  more  generalized  concepts  appear  at  the  highest  level  of  the  map  and  the 
more  specific  concepts  are  arranged  as  subsequent  linkages  (Canas,  Carff,  Hill,  Carvalho,  Arguedas, 
Eskridge,  &  Carvajal,  2005). 

Concept  maps  become  tools  for  building  an  understanding  of  other  cultures  because  each 
manifestation  of  relationships  and  the  propositions  joining  them  bears  the  unique  perspective  of 
the  individual  researcher  or,  in  the  case  of  a  context  inquiry,  of  the  researcher  and  subject.  As 
theoretical  frameworks,  they  have  the  interesting  attribute  that  no  two  maps  are  identical  because 
they  directly  reflect  the  level  of  expertise,  domain  knowledge,  and  linkages  that  the  user  brings  to 
their  construction.  As  an  example,  a  new  laboratory  assistant  will  have  a  very  limited 
understanding  of  the  hierarchical  structure  of  the  entire  research  program,  whereas  a  seasoned 
researcher  will  better  understand  the  interworking  of  the  organization  and  will  have  substantial 
contacts  with  external  organizations  upon  which  the  research  program  relies  for  its  funding.  These 
two  individuals  would  generate  two  vastly  differing  concept  maps:  one  limited  to  the  more  salient 
aspects  of  the  research  work  as  a  whole,  and  the  other  a  more  abstract  representation  of  the 
program  as  but  one  element  in  a  much  larger,  interconnected  network  of  research  organizations 
and  funding  operations.  Thus,  a  concept  map  is  very  much  an  individual  endeavor,  with  many 
different  representations  of  the  interactions  and  interconnections  surrounding  a  knowledge 
domain. 

3.4.  Network  Linkage  Analysis 

Link  analysis  represents  relationships  among  entities  within  a  specified  network  or  within  a 
platform,  such  as  Web  search  engines,  site  visits,  or  telecommunication  networks.  Law 
enforcement  makes  extensive  use  of  link  analysis  to  locate  known  suspects  or  terrorists,  whereas 
financial  organizations  use  link  analysis  to  track  possible  fraudulent  account  transactions. 
Development  of  link  analysis  tools  is  fast  becoming  a  mainstay  for  data  analysts  who  use  these 
tools  to  mine  for  relationships  that  are  purposely  nonlinear  so  to  mask  direct  connectivity.  Figure  6 
shows  an  example  of  a  link  analysis  concerning  a  bank  officer  who  makes  several  loans  to 
customers  with  multiple  fraud  alerts  tied  to  their  respective  bank  accounts.  The  left  side  of  the 
figure  shows  the  network  linkage  diagram  that  identifies  those  accounts  with  alerts  flagging 
fraudulent  activities.  The  right  side  of  the  figure  zooms  in  on  one  individual  (or  nodal  point)  within 
the  network  and  his  relationships  to  these  fraudulent  accounts.  Other  uses  of  link  analysis  include 
monitoring  cyber  security,  mapping  disease  outbreaks,  and  performing  research  on  the  potentials 
of  new  markets. 
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Figure  6.  Network  linkage  analysis  of  fraudulent  transactions.  Image  created  using  Centrifuge 
Systems  Analytics  (2012)  tool. 


Researchers  use  link  analysis  to  identify  central  nodes  within  a  network  and  to  model  the 
interactions  of  those  nodes  across  networks.  Link  analysis  is  also  very  useful  for  recognizing 
patterns  and  trends  within  a  social  network,  and  permits  mapping  of  individuals  and  their 
relationships  within  a  specific  time  window  or  a  defined  geography.  As  a  visualization  framework, 
link  analysis  lends  itself  to  discovery  of  substructures  within  the  larger  networked  environment  and 
allows  an  elaborate  conceptual  mapping  of  the  hierarchy  through  which  nodal  points  interface.  The 
linkages  from  node  to  node  or  across  domains  can  be  defined  based  upon  a  particular  activity  or  to 
track  an  individual  and  their  contacts  to  any  number  of  degrees  from  the  originating  node. 


4.  Visualizing  Quantifiable  Data 

In  developing  the  techniques  and  tools  for  visualizing  social  interactions  and  understanding  the 
underlying  processes,  researchers  begin  with  the  onerous  task  of  selecting  the  data  necessary  to 
achieve  the  stated  objective.  The  advent  of  affordable,  readily  available,  high-capacity  computer 
systems  and  the  accompanying  software  for  data  processing  has  made  the  volumes  of  data 
collected  in  recent  years  truly  staggering.  In  concert  with  affordable,  high-capacity  computing,  data 
analytics  has  enjoyed  a  surge  of  capabilities  brought  on  by  data  mining  algorithms,  self-organized 
mappings,  and  multi-dimensional  visualizations. 

Visualization  of  these  vast  quantifiable  datasets  encompasses  a  broad  epistemological  practice 
categorized  into  three  discrete  actions:  looking,  querying,  and  questioning  (Dodge,  McDerby  & 
Turner,  2008).  Each  of  these  requires  the  researcher  to  select  and  define  the  domain  of  interest  in 
which  to  visualize  the  interactions  among  entities.  Using  a  multiplicity  of  graphic  representations  to 
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address  these  epistemological  practices  has  great  value,  since  it  produces  multiple  viewpoints  of 
the  underlying  data  relationships.  Seeing  data  presented  in  many  different  perspectives  provides 
context  and  aids  the  researcher  in  determining  the  variables  best  suited  for  representing  complex 
social  engagements. 

4.1.  Looking,  Querying,  and  Questioning 

Looking,  querying,  and  questioning  are  unique  components  of  understanding  that  allow  us  to 
conceptualize  the  interrelated  and  sequential  elements  of  the  visualization  process.  These  three 
actions  facilitate  the  development  of  larger  constructs  of  sensemaking  around  which  actors 
engage.  In  the  process  of  understanding  the  cultural  and  social  context  of  a  given  set  of  social 
interactions,  researchers  select,  whether  consciously  or  not,  what  they  perceive  to  be  the  most 
relevant  attributes  of  those  interactions  and  create  visualizations  that  best  portray  how  actors 
engaged  in  these  social  interactions  are  constrained  and  /  or  contextualized. 

Looking  involves  the  observation  of  an  object  of  interest.  The  observer  begins  the  visualization 
process  by  presenting  graphic  depictions  of  an  object.  Looking  entails  simply  defining  an  inquiry 
through  visualization  or  as  a  more  elaborate  activity  in  which  the  observer  defines  the  constraints 
for  observation  (e.g.,  time  of  observation,  duration  of  observation,  scale  of  observation),  the  area 
of  interest  in  which  the  observation  is  conducted,  and  any  signals  that  would  mark  a  change  within 
the  object  under  investigation. 

Querying  establishes  which  data  elements  have  particular  significance  for  understanding  a  social 
dynamic  or  cultural  practice.  Queries  about  these  elements  serve  as  the  direct  interface  that  a 
researcher  deploys  in  assessing  an  event  or  interaction.  Querying  is  a  form  of  classification  that 
introduces  a  wide  range  of  capabilities  and  limitations  on  direct  observation.  By  placing  data  in  pre¬ 
existing  categories,  researchers  open  themselves  to  the  criticism  of  bias  and  selective  visualizations 
that  often  present  prejudicial  views  of  the  actors  being  investigated.  Likewise,  a  poorly  constructed 
query  can  introduce  limitations  with  respect  to  the  data  extracted  for  the  visualization.  Poorly 
constructed  queries  that  are  too  constrictive  of  the  available  data  do  not  necessarily  indicate  a  lack 
of  robustness  within  the  available  data  but  rather  are  more  indicative  of  a  lack  of  understanding  as 
to  how  the  query  is  structured  and  how  the  data  were  collected. 

Questioning  defines  the  very  nature  of  the  investigation.  It  combines  the  visual  insight  rendered 
through  looking  with  the  analytic  results  of  querying  to  explain  the  observed  phenomenon. 
Questioning  represents  the  culmination  of  the  looking  and  querying  processes.  Questions  articulate 
the  discovery  process. 

Interfaces  embedded  in  Web  applications  serve  as  ideal  representations  of  the  visualization 
process  as  a  looking,  querying,  and  questioning  experience.  From  initializing  the  domain  of  interest, 
to  structuring  the  querying  capabilities  of  the  domain,  to  finalizing  the  visualization  and  asking  the 
relevant  why,  what,  and  how  questions,  Web-based  applications  have  proven  invaluable  tools  for 
researchers  culling  a  multitude  of  data  sources.  Using  a  Web  application  interface  as  the  example 
of  looking,  querying,  and  questioning  allows  the  observer  to  examine  output  in  dynamic,  multi- 
formatted  views.  Figure  7  is  a  screenshot  taken  from  the  Spotfire  Decision  Support  System,  an 
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interactive  data  analysis  and  visualization  platform.  The  graphic  provides  a  demographic  snapshot 
of  natality  rates  in  Washington,  D.C.,  showing  percentage  pie  charts  by  gender,  histograms  by  race 
and  ethnicity,  and  a  regression  analysis  of  age  of  mother  vs.  age  of  father. 
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Figure  7.  Natality  data  for  the  District  of  Columbia  as  visualized  from  a  Web-based  database 
application.  Adapted  with  permission  from  Shneiderman  (2013). 


In  Figure  7,  the  multiplicity  of  views  provides  an  overarching  construct  that  helps  researchers  to 
investigate  natality  rates  for  the  city.  By  exploring  parental  characteristics  such  as  racial 
composition,  age,  or  marital  status,  researchers  can  correlate  multiple  factors  influencing  natality 
rates  using  a  multi-view  interface.  For  example,  in  the  above  figure,  the  age  relationship  histograms 
show  that  offspring  of  younger  Black  mothers  in  Washington,  D.C.,  are  more  likely  to  have  older 
fathers  than  the  offspring  of  White,  Asian,  or  Native  American  women.  To  further  explore  this 
pattern,  a  researcher  could  click  on  the  histogram  of  Age  of  Black  Mothers  and  Age  of  Black  Fathers 
to  highlight  those  data  points  within  the  scatterplot  regression  view  and  show  the  variability  of  the 
distribution  against  a  normalized  regression.  Such  visualization  can  be  further  explored  by 
examining  the  highlighted  pie  charts  of  the  age  relationships  by  racial  groups  to  further  investigate 
the  relationship  between  age  of  parents  and  race  of  parents. 

Questioning  structured  hierarchical  relationships  represents  an  important  knowledge  discovery 
technique  and  can  be  used  to  explore  the  underlying  taxonomy  of  the  data.  A  multi-view  interface 
allows  researchers  to  alter  the  classification  schemes  so  as  to  emphasize  particular  relationships 
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considered  more  significant  than  others,  while  also  providing  additional  views  of  the  data  for 
further  analysis  and  hypothesis  testing. 

4.2.  Graphical  Methods  of  Visualizing  Understanding 

Before  delving  into  the  more  sophisticated  techniques  of  visualizing  social  dynamics  and  their 
complex  interactions,  we  first  consider  the  traditional  methods  of  visualizing  metrics  of  the  social 
organization  itself.  These  include  common  graphic  layouts  of  variables  in  such  forms  as  bar  charts, 
scatterplots,  histograms,  and  pie  charts.  Traditional  data  visualizations  such  as  these  allow  an  initial 
glimpse  into  the  underlying  workings  of  organizations,  social  systems,  and  the  actors  engaged 
within  those  relations. 

Charts  present  useful  graphic  displays  of  single  or  multivariate  data  using  simple  x-  and  y-axes  or 
multidimensional  axes  for  variables  whose  distance  measures  depend  upon  the  x  and  y  variables. 
Data  can  be  organized  as  nominal,  ordinal,  interval,  or  ratio.  Charts  can  display  measurements  of 
selected  variables  as  count  values,  average  values,  minimum  and  maximum  values,  median  values, 
or  outliers.  Each  variable  conveys  a  different  perspective  on  the  information  visualized.  Continuous 
measurements  are  useful  in  comparing  fluctuations  across  a  spatial  or  temporal  domain, 
identifying  correlations,  understanding  trends,  and— when  viewed  at  multiple  scales— gaining  a 
holistic  perspective  of  interactions  between  variables.  Discrete  measures  can  be  visualized  as 
categories  of  names,  rankings,  or  scalars. 

Figure  8  displays  population  pyramids  from  the  Chinese  census  from  2000  through  2030.  A 
population  pyramid  amalgamates  census  data  and  visualizes  it  based  upon  two  or  more  variables. 
The  four  population  pyramids  (Figures  8a  through  8d)  show  the  age  shifts  by  gender  beginning  with 
the  2000  census  and  segmented  at  5-year  increments.  In  each  of  these  graphs,  the  population 
distribution  begins  with  the  youngest  group  (0-4  years  of  age)  and  extends  to  100+  years  of  age.  In 
the  2000  and  2010  population  pyramids  (Figures  8a  and  8b),  males  outnumber  females  in  the  age 
categories  0-14.  In  each  of  the  graphs,  10-year  increments  are  demarcated  with  a  darker  bar  than 
the  off-decade  age  groups.  For  both  census  population  pyramids,  the  ratio  of  males  and  females 
remains  equal  until  the  70-and-above  groupings.  The  projected  2020  population  pyramid  (Figure 
8c)  is  the  only  one  to  show  a  larger  proportion  of  the  population  in  the  youngest  age  groups  as 
being  predominantly  female.  Females  outnumber  males  in  the  age  groups  0-24;  the  proportions 
become  equal  at  25  and  show  a  slight  majority  of  males  in  the  60-70  range.  In  the  projected 
population  composition  for  2030  (Figure  8d),  the  ratio  of  males  to  females  increases  dramatically  in 
the  ranges  from  0  to  40  years,  with  females  only  representing  a  majority  of  the  overall  population 
for  ages  60  and  above. 
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Figure  8a  &  8b.  Population  pyramids  of  China,  2000-2030  (estimated)  by  age  groups.  Adapted  from 
the  U.S.  Census  Bureau  (2013),  International  Census  Database. 
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Figure  8c  &  8d.  Population  pyramids  of  China,  2000-2030  (estimated)  by  age  groups.  Adapted  from 
the  U.S.  Census  Bureau  (2013),  International  Census  Database. 
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Understanding  the  demographic  composition  of  a  population  affords  the  researcher  an  overview  of 
the  constitutive  requirements  of  resource  allocation  for  that  society  (Ebenstein,  2010).  The  age 
composition  of  a  population  helps  to  explain  allocation  of  food  and  medical  care  (Marks,  Cargo,  & 
Daniel,  2006).  The  racial,  ethnic,  and  gender  composition  of  a  society  also  provides  insights  into  the 
potential  for  cooperation  or  tensions  within  the  social  network  (Wilson  &  Taub,  2007).  All  of  these 
driving  forces  of  the  social  environment  can  be  appreciated  and  understood  through  the  precursor 
of  the  demographic  composition  that  characterizes  the  society  at  large. 

Conveying  multiple  dimensions  of  information  through  visualization  often  entails  juxtaposition  of 
context  to  further  illustrate  the  graphic  visualizations.  Figure  9  shows  the  global  distribution  of 
private  wealth  from  2009  to  the  estimated  holdings  for  2016.  The  bar  graphs  in  the  figure  are 
divided  into  specific  regions  of  the  world  based  on  geopolitical  boundaries  and  show  wealth  in 
dollars  across  all  private  households.  The  callouts  above  each  year's  bar  graph  indicate  the  percent 
change  in  total  private  wealth  based  on  complete  numbers  and  aggregated  to  the  countries  within 
each  defined  region.  Upon  initial  inspection,  this  graphic  shows  that  private  wealth  in  North 
America,  Western  Europe,  and  Japan  decreased  from  2010  to  2011,  whereas  over  the  same  period 
private  wealth  increased  in  all  the  other  regions  identified.  The  inset  at  the  bottom  right  of  the 
graphic  shows  that  despite  the  decline  in  private  personal  wealth  among  the  wealthiest  nations  of 
North  America,  Western  Europe,  and  Japan,  the  overall  wealth  of  private  households  increases 
steadily  for  each  year  within  the  study  (2009  through  2016  estimated). 
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Figure  9.  Growth  of  global  private  wealth,  2009  to  2016  (estimated).  Adapted  with  permission  from 
the  Boston  Consulting  Group  (2012),  BCG  Global  Wealth  Market  Sizing  Database  2012. 
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Tree  diagrams  (dendrograms)  offer  another  useful  way  to  visualize  hierarchical  relationships 
logically  organized  along  specific  paths.  In  Figure  10,  a  tree  diagram  shows  a  logical  partitioning  of 
the  "Old  World"  (i.e.,  North  America,  Western  Europe,  and  Japan),  which  registered  negative 
growth  of  private  wealth  from  2010  through  2011,  from  that  of  the  "New  World"  (i.e.,  Asia-Pacific, 
Eastern  Europe,  Latin  America,  the  Middle  East,  and  Africa),  which  experienced  significant 
increases  in  private  wealth  during  the  same  period.  The  arrangement  of  the  tree  diagram  gives 
insight  into  the  wealth  distribution  clustered  around  those  regions  that  showed  the  greatest 
increase  in  private  wealth  against  those  regions  with  the  greatest  decrease.  This  arbitrary 
delineation  enables  researchers  to  explore  the  factors  contributing  to  the  "Old  World's"  advances 
and  declines  in  private  wealth  to  those  increases  and  losses  occurring  in  the  "New  World." 
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Figure  10.  Global  Growth  of  "Old  World"  and  "New  World"  Private  Financial  Wealth.  Adapted  with 
permission  from  the  Boston  Consulting  Group  (2012),  BCG  Global  Wealth  Market  Sizing  Database 
2012. 


Reading  a  dendrogram  entails  following  the  path  from  one  level  of  the  hierarchy  to  the  next.  Tree 
diagrams  have  particular  value  in  furthering  understanding  of  complex,  multidimensional 
occurrences  because  they  branch  from  a  known  state  into  unique  and  separate  states  derived  from 
their  original  point  of  departure.  Intelligence  analysts  must  entertain  multiple,  often  contradictory, 
theories  regarding  event  scenarios  in  order  to  eliminate  hypotheses  based  on  potential  outcomes 
derived  from  uncertain  inputs,  and  therefore  find  dendrograms  particularly  useful. 


5.  Visualizing  Understanding  in  Multi-Dimensional  Space 

Computer-based  display  capabilities  make  the  interactions  among  social  actors  more  accessible  by 
presenting  a  dynamic,  multidimensional  perspective.  Social  media  and  interactive  hardware  and 
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software  open  data  exploration  to  a  diverse  range  of  unstructured  data  that  are  relevant  at 
multiple  scales  of  interactions.  Because  social  media  generate  a  "grab  bag"  of  unstructured  data 
streams,  any  traditional  variable  of  interest  in  the  social  sciences  can  be  equated  to  its 
representation  as  a  lumped  parameter  within  social  media.  In  other  words,  some  amalgamation  of 
social  media  data  can  visualize  complex  concepts  such  as  the  political,  economic,  public  health,  or 
demographic  composition  of  a  community.  Similarly,  interactive  media  can  embed  the  observer  in 
controlled  interactions  to  explore  hypotheses  about  engagement  and  create  more  fruitful 
situational  awareness. 

5.1.  Interpreting  Social  Media 

Social  network  analysis  studies  relationships  among  actors  in  a  community  or  group.  While  the 
technology  of  social  networking  is  new,  and  the  human-computer  interfaces  of  social  networks 
continue  to  evolve,  the  messaging  and  social  structures  reified  in  this  technology  are  of  keen 
interest  to  the  social  and  behavioral  sciences  for  the  same  reasons  that  one-on-one  interviews 
remain  a  relevant  data  collection  method:  human  interactions  are  complex  and  multidimensional, 
and  are  based  upon  social  organizations  and  cultural  practices.  The  virtual  space  offered  through 
social  media  is  the  quintessential  multidimensional  space.  Social  media  allows  users  to  interact 
with  individuals  across  the  globe,  engage  in  activities  that  span  the  gamut  of  interests,  and  allow 
one  to  assume  an  identity  consummate  with  one's  personal  interests. 

Social  network  analysis  came  into  its  own  in  the  1960s.  The  process  is  built  on  network  graph 
theory,  in  which  complex  interactions  are  visualized  as  networks.  Network  graph  theory  visualizes 
connections  using  a  standard  set  of  tools  that  include  nodes  represented  as  circles  or  ovals  and 
linkages  represented  as  arrows.  The  relative  size  and  shape  of  a  node  or  vertex  indicate  the  node's 
or  vertex's  prominence  within  the  network.  The  lines  linking  vertices  represent  directed  or 
undirected  linkages,  depending  upon  how  individual  entities  relate  to  each  other  within  the 
network.  A  directed  link  indicates  a  reciprocal  relationship;  for  example,  the  way  "friends"  link  to 
each  other  in  the  Facebook  network.  An  undirected  link  represents  a  user  following  another  user, 
however,  an  undirected  link  relationship  is  not  necessarily  reciprocal.  For  example,  a  Twitter  user 
may  follow  a  very  popular  Twitter  account,  such  as  that  held  by  a  celebrity  or  politician,  but  need 
not  engage  with  the  account  owner  to  be  part  of  that  network. 

Social  networks  are  of  great  interest  to  social  scientists  because  the  structure  of  the  network 
visualization  provides  information  about  the  activity  surrounding  particular  nodes,  not  just  about 
the  nodal  point  itself.  The  density  of  arrows  to  a  particular  node  indicates  the  popularity,  or 
prominence,  of  an  entity  within  the  network.  Shorter,  more  densely  drawn  arrows  indicate 
relationships  between  nodes  that  entail  significant  bi-directional  traffic.  Networks  emerge,  gain 
prominence,  motivate  actions,  and  /  or  influence  events.  A  research  area  that  will  prove  significant 
is  to  chart  the  ecosystem  of  a  social  network,  e.g.,  how  does  an  influential  social  network  emerge? 
What  are  the  characteristics  informing  its  expansion?  What  impacts  does  a  social  network  have  on 
its  members?  Flow  does  a  social  network  continue  to  foster  relevance  or,  consequently,  what  are 
the  determining  factors  in  its  decline? 
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5.2.  Online  Social  Networks 

Online  social  networks  include  email  networks,  discussion  boards,  threaded  conversation 
networks,  Twitter,  Facebook,  YouTube,  Flickr,  and  the  ever-expanding  hyperlink  networks  of  the 
World  Wide  Web  (WWW).  These  networks  connect  disparate  groups  through  the  confluence  of 
influential  contributors,  timely  topics,  and  interests  in  furthering  a  particular  cause.  Online 
communities  emerge  through  any  number  of  configurations  constructed  on  the  basis  of  political, 
social,  economic,  ethnic,  religious,  or  other  impulses.  Figures  11a  and  lib  show  Smith's  (2013) 
comparative  analysis  of  the  social  network  that  emerged  during  the  "Occupy  Wall  Street"  protests 
using  a  snapshot  of  Twitter  tweets  on  November  15,  2011,  and  network  traffic  on  the  same  day 
among  Tea  Party  adherents.  The  respective  networks  are  grouped  by  their  affinities  and  show 
undirected  links,  or  "replied  to,"  "mention,"  or  "follow,"  to  either  "occupywallstreet"  or  "teaparty." 
Replies  and  mentions  stream  across  various  groups  within  each  network. 

Of  particular  interest  in  contrasting  these  networks  is  that  at  the  time  of  Occupy  Wall  Street's 
greatest  momentum  in  November  2011,  the  majority  of  network  traffic  involved  replies  and 
mentions,  whereas  in  the  Tea  Party  network  the  majority  of  traffic  flowing  among  the  affiliated 
nodes  involved  following  particular  accounts,  with  far  less  direct  engagement  of  isolated  users.  An 
observer  noting  the  characteristics  of  these  respective  networks  would  conclude  the  cohesiveness 
of  Occupy  Wall  Street  to  be  far  more  entrenched  than  those  of  their  Tea  Party  counterparts.  Yet, 
the  organizational  staying  power  of  the  Tea  Party  network  suggests  that  while  the  community 
interactions  of  Occupy  Wall  Street  were  more  densely  construed,  the  adherence  to  the  social 
message  of  a  few  influential  members  of  the  Tea  Party  network  is  able  to  reverberate  throughout 
the  social  fabric  and  remain  a  relevant  part  of  the  political  discourse. 
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Figure  11a.  Social  network  comparison  of  Occupy  Wall  Street  and  Tea  Party  -  Occupy  Wall  Street 
network.  Adapted  with  permission  from  The  Social  Media  Research  Foundation  (2012),  NodeXL 
Network  Graphs. 
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Figure  lib.  Social  network  comparison  of  Occupy  Wall  Street  and  Tea  Party  -  Tea  Party  network. 
Adapted  with  permission  from  The  Social  Media  Research  Foundation  (2012),  NodeXL  Network 
Graphs. 


During  the  height  of  media  attention  focused  on  Occupy  Wall  Street,  the  network  comparisons 
suggested  Occupy  Wall  Street  had  a  greater  community  footprint  than  the  Tea  Party.  However,  a 
social  researcher  may  be  particularly  interested  in  the  long-term  configurations  of  these  respective 
social  networks  as  they  experience  vastly  different  political  lifecycles. 

Social  networking  sites  such  as  Facebook  and  Twitter  have  millions  of  subscribers  who  post  daily 
updates  regarding  their  personal  activities,  relationship  connections,  and  general  interests.  This 
information  is  available  to  members  of  each  participant's  personal  network.  Social  communities 
exist  within  the  confines  of  these  social  networking  platforms.  These  communities  can  openly 
express  the  ideas  that  encompass  their  notion  of  community  or  can  be  more  secretive  and  actively 
mask  their  community's  underlying  objectives.  Social  communities  that  use  network  sites  to  grow 
and  to  propagate  their  ideas  supply  intelligence  analysts  with  a  wealth  of  information  on  the  topics 
that  define  or  directly  impact  their  community. 

Social  network  analysis  tools  are  evolving  to  include  sophisticated  linguistic  analysis  of  sentiment 
conveyed  in  the  messages  that  comprise  the  overwhelming  traffic  on  social  networking  sites.  While 
the  inclusion  of  linguistic  analysis  of  social  media  is  a  new  venue  in  social  research,  it  promises  a 
tremendous  value  in  assessing  the  state  of  a  given  community  and  how  they  perceive  their 
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interactions  among  themselves  and  with  other  groups.  In  the  following  sections  we  explore  two 
social  media  analysis  tools  used  to  assist  intelligence  analysts  in  understanding  the  sentiment 
surrounding  a  topic  or  event:  Tweet  Explorer  using  Linguistic  Inquiry  and  Word  Count  (UWC)  and 
Sentimedir. 

5.2.1.  Tweet  Explorer/  Linguistic  Inquiry  and  Word  Count 

The  MITRE  Corporation  developed  Tweet  Explorer  as  a  high  level  integration  tool  that  tracks  the 
general  mood  of  Twitter  users  and  their  interests  in  domestic  and  international  social  and  cultural 
events.  Analysts  query  tweets  based  on  semantic  parameters,  including  hash  tags,  keywords, 
locations,  and  dates.  Tweet  Explorer  then  sorts  key  words  identified  within  the  tweets  using 
indicator  words  defined  by  the  UWC  dictionary.  Indicator  words  connote  emotional  state,  such  as 
satisfaction,  anger,  happiness,  etc. 

Tweet  Explorer  visualizes  trends  in  the  mood  reflected  in  the  extracted  tweets  by  mapping  levels  of 
emotions  expressed  across  a  timeline  and  allowing  users  to  discover  the  times  at  which  levels  of 
emotions  shifted  significantly  into  a  higher  or  lower  phase.  Once  these  'breakpoints'  in  time  have 
been  identified,  users  can  examine  them  against  events  that  unfolded  on  the  ground  at  those 
times.  In  this  way,  users  can  gain  insight  into  the  emotions  people  expressed  while  certain  events 
occurred.  After  isolating  a  set  of  tweets,  the  analyst  can  view  the  actual  text  of  the  tweet  for 
further  consideration. 

In  Figure  12,  a  timeline  represents  the  results  of  a  query  coincident  with  the  pending  national 
election  in  Kenya.  This  example  shows  the  time  period  January  23,  2012  to  July  2,  2012.  During  this 
time,  the  political  leaders  in  Kenya  and  much  of  the  international  community  braced  themselves 
for  potential  outbreaks  of  tribal-induced  violence  reminiscent  of  the  aftermath  of  the  national 
elections  in  2007.  The  scatter  plot  shows  levels  of  anger  expressed  in  tweets  referring  to  the 
national  elections  and  the  various  political  parties.  Note  the  sharp  increase  around  January  23. 
During  this  time,  the  International  Criminal  Court  formally  charged  Deputy  Prime  Minister  and 
presidential  candidate  Uhuru  Kenyatta  along  with  member  of  Parliament  William  Ruto  of  crimes 
against  humanity.  Subsequent  outbursts  of  anger  appear  at  the  end  of  April  2012,  which  coincides 
with  a  church  bombing  in  Nairobi. 
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Figure  12.  Tweet  Explorer  output  of  anger  sentiments  in  Kenya,  January  to  July  2012.  Adapted  with 
permission  from  the  MITRE  Corporation  (2012),  Social  Radar  Research  Group. 


5.2.2.  Sentimedir 

As  social  media  proliferate  and  transition  from  personal  chat  forums  and  community  share  sites  to 
principal  sources  of  information  and  commentary  on  cultural  events  happening  "in  the  street," 
decision  makers,  be  they  city  planners  soliciting  feedback  on  proposed  biking  lanes  in  a 
metropolitan  area  or  marketing  experts  seeking  to  identify  the  next  big  thing,  recognize  these  new 
media  formats  as  potential  gauges  of  how  sentiments  within  groups  unfold  in  near-real  time.  By 
isolating  the  preponderance  of  targeted  words  and  phrases  conveying  emotions  associated  with  a 
given  topic  and  the  behavior  of  individuals  in  response  to  events  of  interest,  social  media  has  the 
potential  to  revolutionize  how  groups  respond  to  and  interact  as  events  unfold.  This  approach  to 
social  engagement  analysis  focuses  on  the  language  used,  the  cathartic  impulses  conveyed  and  the 
impact  of  coordinated  responses  as  exposed  in  the  social  messages  that  proliferate. 

The  MITRE  Corporation  developed  a  prototype  tool— Sentimedir— to  help  gauge  the  operational 
value  of  sentiment  analysis.  Sentimedir  ingests  text,  primarily  from  Web-based  sources,  including 
news  and  social  media.  It  applies  natural  language  processing  (NLP)  techniques  to  exploit 
semantic,  lexical,  and  syntactic  features  in  order  to  automatically  detect  expressed  sentiment 
across  millions  of  documents.  System  users  can  apply  complex  Boolean  combinations  of  free  text 
queries  and  structured  filters  against  this  processed  data  store  to  help  illuminate  the 
preponderance  and  dynamics  of  sentiment  relative  to  many  different  features  of  the  messages, 
including  publication  source,  country,  topic,  date  or  even  individual  authors  or  detected  opinion 
holders.  Users  may  then  visualize  aggregate  results  in  a  variety  of  ways,  including  a  daily  index  of 
overall  sentiment,  presented  as  a  moving  average  with  discrete  indicators  signaling  when  a  change 
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in  the  index  is  statistically  significant  (Day,  Boiney,  Brown,  Quimby,  &  Ubaldino,  2010).  Other 
visualizations  include  stacked  bar  charts  of  the  volume  of  positive  and  negative  sentiment  relative 
to  these  same  kinds  of  metadata  features,  and  pie  charts,  histograms  and  "word  clouds"  to  display 
the  various  absolute  and  relative  quantities  of  these  different  kinds  of  phenomena.  In  effect,  this 
is  a  sentiment  sensor  capable  of  detecting,  measuring,  and  contextualizing  the  extent  to  which  a 
sentiment  is  shared  within  a  group  of  users.  The  Sentimedir  prototype  is  one  of  many  research  and 
development  efforts  underway  to  test  the  broad  hypothesis  that  applied  sentiment  analysis 
technology  can  be  an  effective  tool  for  visualizing  and  analyzing  social  engagement. 

5.3.  Virtual  Reality  Visualizations 

Visualization  techniques  that  move  beyond  two-  or  three-dimensional  graphic  representations  to 
more  interactive,  immersive,  multidimensional  analytic  space  further  aid  understanding  of  the 
social  dynamics  of  individual  and  group  behaviors  across  cultural  and  social  domains.  Software 
packages  have  not  only  advanced  functional  capabilities  for  modeling  complex  social  interactions 
but  can  also  immerse  the  observer  in  high  fidelity  virtual  realities  that  simulate  the  actual 
environment  of  interest.  In  addition,  advances  in  hardware  have  significantly  increased  the  degree 
to  which  observers  can  embed  themselves  in  a  scene,  which  provides  an  unencumbered  sense  of 
interaction.  These  immersive  technologies  can  potentially  allow  users  to  play  out  a  multiplicity  of 
scenarios  based  upon  research  conducted  on  the  ground  with  individuals  from  the  social  and 
cultural  groups  of  interest,  as  discussed  in  the  previous  sections. 

5.3.1.  Surrogate  travel 

Surrogate  travel  combines  the  display  of  discrete  variables  underlying  the  foundation  of  the 
"scene"  with  interactive,  multimedia  simulations  of  an  observer's  experience  of  the  created 
environment.  It  integrates  observer  participation  into  the  sensemaking  process,  but  does  not 
immerse  the  observers  in  a  virtual  environment;  a  significant  distance  remains  between  the 
observers  and  the  environment  in  which  they  are  engaged.  Surrogate  travel  can  take  the  form  of 
simulated  flight  panels,  video  cameras  providing  live  feeds  of  a  cruise  ship  or  traffic  flow,  or  even  a 
live-camera  feed  of  animals  at  the  zoo  that  allows  observers  to  check  on  the  newest  additions  to 
the  institution. 

Flight  simulators  offer  excellent  examples  of  surrogate  travel.  The  more  recent  simulators  combine 
visual  effects,  sound  effects,  air-to-ground  communication,  and  full  body  motion  as  part  of  the 
experience  a  pilot  would  anticipate  in  a  cockpit.  The  simulated  cockpit  moves  in  the  direction  of  the 
turns  executed  by  the  pilot  at  the  control  panel,  and  the  throttle,  rudders,  and  yoke  exert  a 
counterforce  on  the  system.  The  plane's  movements  modulate  the  simulation  of  the  terrain  that 
the  aircraft  traverses  as  a  function  of  increasing  or  decreasing  altitude.  While  the  plane  is  in 
simulated  motion,  pixelated  views  of  the  landscape  alter  as  a  function  of  the  plane's  threshold  with 
respect  to  the  radiance  angle,  while  landscape  shadowing  reflects  the  zenith  angle  of  the  sun's  rays 
based  on  the  specific  times  of  day  programmed  into  the  simulation.  Engines  and  instrumentation 
sounds  are  integrated  into  the  simulation,  as  are  the  communication  relays  with  the  air  traffic 
control  tower.  This  type  of  simulation  familiarizes  pilots  with  the  way  controls  respond  to  their 
input  and  with  the  terrain  through  which  they  will  fly,  and  helps  them  appreciate  the  sound 
patterns  within  the  cockpit  and  the  communication  relays  they  will  use. 
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5.3.2.  Semi-immersive  animation  environments 

Virtual  reality  modeling  languages  (VRMLs)  (Cartwright,  2008)  enable  the  technical  development  of 
semi-immersive  and  fully  immersive  animation.  In  these  virtual  worlds,  the  user  adopts  an  online 
presence,  or  avatar,  who  becomes  their  persona  in  any  number  of  virtual  environments.  Second 
Life,  the  online  immersive  space,  is  an  example  of  a  virtual  world  in  which  users  create  their  living 
environments,  establish  relationships  with  other  avatars,  and  generate  a  fully  functioning  second, 
virtual  world.  King  and  Fouts  (2009)  used  Second  Life  as  a  segue  into  understanding  the  cultural 
practice  of  ijtihad2  in  the  larger  context  of  understanding  Islam  from  a  Western  perspective.  In 
their  virtual  journey  to  the  Hajj  in  Mecca,  the  two  investigators  encountered  Muslims  from  around 
the  world  with  whom  they  were  able  to  pose  questions  regarding  Islamic  culture  and  practices.  By 
developing  their  avatars,  the  researchers  were  able  to  adopt  the  traditional  dress  and  ornaments 
of  the  Hajj  and  gain  a  better  understanding  of  what  this  ritual  means  to  practitioners  around  the 
world. 

Semi-immersive  animation  is  used  at  the  University  of  Southern  California  (USC)  Institute  for 
Creative  Technologies  (ICT)  to  create  a  training  theater  in  which  troops  can  simulate  direct 
interactions  with  villagers,  patrol  streets  known  to  be  populated  with  snipers,  and  experience  the 
chaotic  engagement  of  a  simulated  street  battle  that  includes  the  sounds  of  enemy  fire, 
shockwaves  from  Improvised  Explosive  Devices  (lEDs),  and  the  deafening  sounds  of  helicopters 
hovering  above  the  battlefield.  In  preparation  for  strategic  missions,  virtual  reality  is  used  to 
prepare  personnel  for  deployments  into  unfamiliar  terrain.  Virtual  reality  simulations  of  combat 
actions  help  train  troops  about  battlefield  conditions  and  better  familiarize  them  with  the  pace  of 
engagement  and  the  confusing  interactions  inherent  in  actual  combat  (Vasquez,  2008). 

Figure  13  is  an  example  of  this  new  kind  of  training  environment  known  to  researchers  at  ICT  as 
FlatWorld.  FlatWorld  exposes  combat  troops  to  the  look  and  feel  of  an  Afghan  village.  In  FlatWorld, 
all  aspects  of  a  battle  are  virtualized,  including  the  helicopter  flying  overhead,  which  actually 
triggers  shockwaves  through  the  enclosure's  floor  when  passing  through  the  scene.  Scents  can  also 
be  added  to  the  environment  to  simulate  the  smell  of  gunpowder,  diesel  fuel,  and  even  the 
vegetables  and  flowers  from  the  produce  stall  across  the  street.  While  this  technology  advances 
real-time  simulation  of  battle  conditions,  the  experience  is  not  fully  immersive  because  the 
observers  remain  outside  the  scene,  and  colleagues  or  subject  matter  experts,  also  outside  the 
theater's  enclosure,  direct  them  on  how  to  adjust  their  responses  to  the  unfolding  scenario. 


2  Ijtihad  is  the  practice  of  independent  interpretation  of  Islamic  law  (Sharia)  based  upon  a  methodic  reasoning 
of  legal  precedence  and  couched  in  a  practiced  understanding  of  Islamic  theology,  a  scholarly  appreciation  of 
the  Quran,  the  traditional  stories  of  the  Prophet's  life  (Hadith),  and  scholastic  consensus  (ijma).  For  further 
discussion  of  Ijtihad  see  Gesink  (2003). 
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Figure  13.  Semi-immersive  training  scenario  for  combat  in  an  Afghan  village.  Adapted  with 
permission  from  the  Institute  for  Creative  Technologies  (2007). 


5.3.3.  Fully  immersive  animation  environments 

Visualization  of  virtual  environments  is  not  constrained  to  simulation  on  a  computer  monitor. 
Many  military  and  academic  research  centers  have  developed  virtual  environment  tools  that  fully 
envelop  the  subject  in  a  virtual  reality  experience. 

Full  immersion  animation  can  be  realized  as  an  entire  room  designed  as  a  high-resolution,  multi- 
sensory  simulation  theater  capturing  user  interactions  through  real-time  scenarios.  Researchers  at 
the  University  of  Illinois  at  Chicago  designed  and  built  the  Cave  Automatic  Virtual  Environment 
(CAVE),  which  consists  of  a  room-sized  3-D  video  and  surround-sound  audio  system  with  graphics 
projected  on  three  walls  and  the  floor  (Kenyon,  1995).  Users  interact  with  the  technology  by 
wearing  a  location  sensor  and  stereo  vision  googles.  The  location  sensor  signals  to  the  3-D  video 
and  audio  playback  to  adjust  viewing  perspective  and  stereo  projection  relative  to  the  user's 
location  in  real-time  while  the  stereo  vision  googles  allow  the  user  to  experience  the  3-dimensional 
visualization  at  close  range.  The  CAVE  is  designed  for  scientific  visualization  and  has  been  used  to 
visualize  such  disparate  events  as  simulating  displacements  along  a  geologic  fault  plane  or  the 
buildup  and  dispersion  of  a  water  wave  within  a  confined  basin.  Entire  virtual  ecosystems  can  be 
visualized  using  a  fully  immersive  environment  as  evidenced  by  the  burst  of  film  projects  utilizing  3- 
D  technology  to  animate  these  interactive  virtual  terrains. 
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6.  Visualizing  Understanding  Using  Big  Data  Analytics 

"Big  data"  is  a  term  of  art  that  includes  both  the  processing  of  massive  data  stores  and  the 
technological  advances  that  in-memory  data  systems  and  storage  capabilities  require  to  fully 
ingest,  process,  and  analyze  vast  data  streams.  LeHong  and  Fenn  (2012)  refer  to  big  data  as  a 
three-dimensional  problem:  increasing  volumes  of  data  (several  terabytes  to  petabytes  of  data); 
the  speed  with  which  the  data  must  be  collected,  processed,  and  delivered;  and  the  multiplicity  of 
data  formats.  This  is  known  as  the  "3Vs"  model:  volume,  velocity,  and  variety.  While  new  data 
sources  and  modeling  techniques  offer  greater  possibilities  for  visualization  and  understanding, 
these  same  sources  and  techniques  expose  the  researcher  to  new  challenges  that  call  for  new  ways 
of  conceptualizing  analysis  and  visualization,  as  well  as  the  need  for  tools  to  integrate  these 
techniques  within  the  overall  sensemaking  enterprise. 

The  limitations  previously  imposed  on  the  size  of  datasets  and  the  kinds  of  data  types  that  could  be 
processed  are  rapidly  disappearing.  Data  are  no  longer  collected  and  siloed  on  physical  storage 
systems,  and  researchers  no  longer  need  physical  access  to  the  storage  medium  or  hardware  that 
enables  persistent  connections  to  the  siloed  storage.  It  is  not  coincidental  that  descriptions  of  data 
access  in  virtual  space  use  water  metaphors  to  describe  the  experience:  data  streams,  data  flows, 
the  fire  hose  of  information,  etc.  These  new  ways  of  conceptualizing  the  very  access  to  data  enable 
previously  unimaginable  possibilities  for  new  technologies  to  analyze,  visualize,  and  test  research 
hypotheses. 

The  social  science  community  has  yet  to  fully  realize  the  analytic  potential  that  big  data  may  offer. 
Here  we  examine  those  analytic  capabilities  most  applicable  to  visualizing  understanding  within  the 
social  context.  These  capabilities  include  analytics  of  social  networks  using  big  data,  cross-channel 
analytics,  emotion  and  sentiment  detection,  and  agent-based  modeling  to  advance  intelligence 
analysis  of  crowd  behavior. 

6.1.  Social  Network  Analysis  Using  Big  Data 

Big  data  describes  much  of  the  social  media  output  and  includes  real-time  analysis  of  volumes  of 
Twitter  feeds,  Facebook  updates  and  photo  posts,  blog  posts,  video  feeds,  and  satellite  streams. 
While  the  popularity  of  these  social  media  sites  is  astonishing  and  continues  to  expand,  the 
analytical  community  continues  to  debate  the  most  advantageous  means  to  collect  and  analyze 
these  data.  Preis,  Moat,  Stanley,  and  Bishop  (2012)  give  an  example  of  analyzing  Web  search  logs 
using  Google  Trends  to  identify  potential  patterns  in  the  data  and  found  a  unique  correlation 
between  online  behavior  and  the  economic  indicators  of  the  country  in  which  the  user  resides. 
Their  study  clustered  Internet  searches  into  those  including  search  terms  of  '2009'  and  earlier  and 
those  with  '2011'  and  later.  The  research  found  that  Internet  users  in  wealthy  countries  performed 
far  more  future-oriented  searches  than  Internet  users  in  countries  with  low  gross  domestic  product 
(GDP).  Users  in  those  countries  searched  more  often  on  past  events  (Figure  15). 
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Figure  15.  Internet  search  of  the  future  vs.  the  past  based  on  country  GDP.  Adapted  with 
permission  from  Preis,  Moat,  Stanley,  &  Bishop  (2012). 


While  such  results  can  be  construed  as  a  discovered  correlation,  determining  whether  they  indicate 
cultural  trends  or  are  merely  a  happenstance  of  the  experimental  design  remains  an  ongoing  issue 
in  big  data  analysis.  Because  the  data  are  by  and  large  unstructured,  forcing  structure  onto  the  data 
can  mask  actual  relationships  while  simultaneously  imposing  artificial  trends  or  correlations  that 
are  artifacts  of  the  data  processing  routines  used  to  filter  the  data. 

The  data  used  to  generate  Figure  15  provide  a  snapshot  of  log  files  for  websites  throughout  the 
Internet.  Analyses  of  these  data  demand  that  researchers  cull  massive  data  sets,  extract 
meaningful  relationships,  and  assess  the  effectiveness  of  those  relationships.  However,  big  data 
also  affords  researchers  the  possibility  to  further  hypothesize  and  visualize  these  data  to  include 
geographic  determinates  of  these  more  future-oriented  countries.  For  example,  do  specific  future- 
oriented  ideas  link  Germany,  Britain,  France,  and  Italy  while  simultaneously  rendering  the 
relationships  between  the  United  States,  Canada,  and  Japan  more  restricted?  Is  Russia  a 
transitional  country  moving  from  a  past-oriented  society  to  one  that  is  predominantly  future- 
oriented— and,  if  so,  what  driving  forces  within  this  society  are  causing  this  shift  of  perspective? 
Such  pattern  recognition  can  lead  to  new  insights  into  the  inter-workings  of  these  countries  as  part 
of  the  massive  overhaul  of  social  institutions  they  are  currently  experiencing. 

6.2.  Cross-Channel  Analytics  with  Emotion  Detection 

Cross-channel  analytics— the  study  of  interactions  across  organizations  or  across  different  media 
channels  (e.g.,  print,  audio,  websites,  or  mobile  devices)— uses  big  data  to  track,  link,  and  analyze 
social  actors  and  their  interactions.  Tracking  an  individual's  behavior  across  a  multiplicity  of 
communication  channels  is  not  new  in  social  behavioral  research;  however,  using  analytics  to 
amass  volumes  of  varied  data  and  process  these  data  at  the  computational  speeds  afforded  by 
high-end  computing  systems  will  open  a  new  field  of  investigation  with  respect  to  understanding 
and  visualizing  the  underlying  social  dynamics. 
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In  conjunction  with  cross-channel  analytics,  researchers  need  new  tools  to  exploit  and  better 
understand  sentiment  among  social  actors.  Detection  of  an  individual's  emotional  state  (e.g., 
anger,  confusion,  happiness)  through  voice  channels  illustrates  how  social  science  researchers  can 
use  the  new  tools  and  technologies  to  further  explore  dynamic  social  interactions.  Sentiment 
analysis  applies  equally  to  verbal  messaging  and  text  messaging;  as  these  new  detection 
technologies  become  available,  the  interrelationships  among  emotional  connections,  sentiment 
analysis,  and  social  interactions  will  be  more  readily  accessible. 

Currently,  emotion  detection  is  a  key  underlying  variable  in  training  physicians  to  treat  patients 
suffering  from  post-traumatic  stress  disorder  (PTSD),  especially  patients  who  perceive  the 
condition  as  a  personal  failing.  The  Defense  Advanced  Research  Programs  Agency  (DARPA) 
Detection  and  Computational  Analysis  of  Psychological  Signals  (DCAPS)  project  applies  machine 
learning,  NLP,  and  computer  vision  to  generate  training  scenarios  for  physicians.  In  these  scenarios, 
the  virtual  patient  displays  a  range  of  facial  expressions  in  an  attempt  to  mask  symptoms  or 
deceive  the  physician  regarding  his  or  her  emotional  state.  This  simulation  technology  enables 
DCAPS  to  program  the  virtual  patient  to  allude  to  signs  of  PTSD  or  display  emotional  clues 
indicative  of  PTSD  that  the  physician  learns  to  recognize  during  a  medical  evaluation. 

Equally  compelling  is  that  the  virtual  patient  can  be  programmed  in  real  time  using  an  amalgam  of 
sensors  that  simultaneously  perceive  the  physician's  body  language  and  physical  signals  during  the 
simulated  diagnostic  session.  The  virtual  patient  can  modify  his  or  her  responses  based  on  the 
physician's  level  of  ease  or  discomfort  given  the  context  of  interactions.  In  this  way,  physicians 
become  aware  of  their  own  influence  on  the  diagnostic  interview  and  how  their  actions  may  either 
help  or  hinder  patients  in  discussing  their  symptoms. 

6.3.  Gaps  in  Visualization  of  Understanding 

Social  analytics  of  big  data  has  tremendous  potential  for  predictive  analysis  and  modeling  of 
associated  trends.  The  patterns  and  connections  uncovered  using  big  data  can  be  disseminated  in 
near-real  time  to  subsequent  models  designed  to  address  specific  behaviors  or  associations.  For 
example,  business  intelligence  tools  can  be  integrated  with  data  mining  algorithms  to  foster 
automated  discovery  and  generate  output  in  formats  such  as  near  real-time  simulations, 
increasingly  complex  semi-  and  full-immersion  animations,  and  more  granular  summary  tables 
based  upon  hitherto  unknown  associations.  Filtering  of  Internet  search  terms  using  link  analysis,  for 
example,  could  reduce  the  nodes  in  a  social  network  to  those  representing  specific  behaviors 
among  actors  with  particular  affiliations. 

Adoption  of  social  network  analysis,  however,  has  been  hampered  by  the  perception  that  the 
process  is  highly  abstract  and  that  the  information  collected  is  difficult  to  translate  into  intelligible 
actions.  Further,  the  design  of  association  algorithms  and  search  indexing  routines  can  lead  to 
questionable  methods  that  may  cause  researchers  to  draw  conclusions  based  upon 
misunderstandings  of  the  scope  and  accuracy  of  the  methodology.  Moreover,  lack  of  experience 
with  big  data  highlights  the  need  for  well-defined  data  governance  at  all  levels  of  research,  and  for 
standards  and  best  practices  to  guide  the  implementation  of  big  data  in  the  social  sciences. 
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Considerable  barriers  still  limit  the  uses  of  big  data  and  the  tools,  processing  modes,  and  modeling 
capabilities  that  big  data  promises.  For  example,  the  lack  of  sufficient  analytic  capabilities  to  mine 
social  media  for  relevant  patterns  and  correlations  against  background  noise  presents  a  significant 
hurdle  to  the  integration  of  big  data  visualization  practices.  While  big  data  opens  a  new  field  in 
data  processing  and  visualization,  the  very  tools  for  modeling  and  visualizing  this  output  require 
new  modes  of  conceptualizing  behavioral  modeling  as  a  practice.  Today's  challenge  in 
distinguishing  the  signal  from  the  noise  is  only  amplified  when  static  datasets  become  part  of  a 
data  stream. 

Aside  from  the  technical  requirements  of  new  modeling  methodologies,  concerns  related  to 
privacy  rise  to  a  new  level  with  respect  to  big  data.  General  users  of  social  media  may  come  to  feel 
so  constricted  in  their  use  of  the  media  by  extensive  online  profiling  that  they  may  choose  to 
abandon  the  media  altogether.  Conversely,  users  who  know  how  they  are  expected  to  behave  may 
begin  using  social  media  to  better  align  their  online  presence  with  expected  behavior.  In  addition, 
aggregating  data  about  individual  behaviors  based  on  an  Internet  search  index  is  not  the  equivalent 
of  social  behavior.  Researchers  need  to  be  sensitized  to  better  appreciate  the  nuances  of  this  new 
data  environment. 

Another  significant  factor  preventing  the  full  integration  of  big  data  in  understanding  and 
visualizing  social  behavior  is  the  role  that  anonymization  (the  processing  steps  used  to  obfuscate  an 
individual's  identity  during  data  analysis)  plays  in  protecting  individual  identities.  Online  privacy 
and  corporate  liability  in  the  event  of  a  security  breach  or  a  leak  of  private  information  have 
received  considerable  attention,  but  the  research  community  does  not  widely  appreciate  that 
while  the  principles  of  privacy  are  considered  sacrosanct,  the  means  for  ensuring  that  privacy  are 
not  foolproof  (Ohm,  2009).  No  one  algorithm  or  method  suffices  to  fully  anonymize  data.  As  in 
other  areas  in  which  a  particular  information  item  may  be  considered  low  sensitivity  by  itself,  the 
information  may  become  much  more  revealing  when  low  sensitivity  information  is  merged  with 
other  data. 


7.  New  Visualization  and  Predictive  Modeling 

New  models  for  scientific  visualization  have  emerged  that  exploit  the  potentials  of  big  data  and 
promise  to  circumvent  some  of  the  previously  discussed  limitations  of  traditional  data  querying 
algorithms  (Purchase,  2008).  These  new  visualization  methods  must  address  data  so  massive  that 
simply  framing  the  problem  to  be  solved  can  very  well  hinder  pattern  discovery.  Data  as  a  scientific 
inquiry  unto  itself  and  the  visualization  of  data  as  a  dynamic  process  are  new  ways  of  considering 
the  role  of  data  in  knowledge  discovery  and  acquisition. 

This  chapter  has  focused  on  visualizing  the  underlying  baseline  dynamics  that  increase 
understanding  of  social  conditions  and  interactions.  The  technological  breakthroughs  of  big  data 
can  better  inform  predictive  models  of  social  behaviors  and  provide  more  succinct  visualization  of 
multidimensional  data  outputs.  Agent-based  modeling  offers  one  example  of  coupling  the 
requirements  of  vast  data  processing  to  the  visualization  of  dynamic  interactions  at  multiple  scales. 
Combining  agent-based  modeling  with  big  data  analytics  to  generate  self-organizing  maps  can 
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further  contribute  to  meeting  the  research  goal  of  integrating  individual  behavioral  dynamics  into 
group  dynamics  across  a  specified  domain,  whether  that  movement  occurs  in  a  social  network  or  a 
physical  environment. 

7.1.  Agent-Based  Modeling 

Greater  reliance  on  high-speed,  parallel  processing  environments  and  vast  improvements  in  virtual 
storage  capabilities  give  big  data  a  tremendous  advantage  over  the  traditional  database 
approaches  of  pattern  recognition,  cluster  analysis,  and  correlation  studies.  Coupling  agent-based 
modeling  with  big  data  gives  researchers  a  powerful  set  of  tools  to  model  human  behavior  and 
constrain  that  behavior  by  environmental  or  temporal  boundaries.  Agent-based  modeling 
represents  a  new  approach  to  generating  data  for  visualization  made  possible  by  these 
environments.  In  this  dynamic  modeling  process  interactions  on  a  microscopic  scale  are  summed  to 
the  interactions  that  comprise  the  group  dynamics.  An  agent-based  model  includes  a  population  of 
individual  "actors,"  with  the  number  of  actors  conditioned  by  the  needs  of  the  model.  Each  actor  is 
implemented  as  a  distinct  data  structure  with  unique  attributes  specified  by  the  model.  Successive 
iterations  of  actions  between  individual  actors  and  among  groups  of  individuals  build  macroscopic 
regularity.  While  discrete  collections  of  actors  may  perform  in  a  somewhat  homogeneous  manner 
(group  behavior),  the  actors  are  heterogeneous  in  relation  to  other  groups  and  behave  according 
to  the  dynamics  of  human  error,  peer  pressure,  and  biased  reasoning  (Tsang,  2008). 

Agent-based  modeling  also  generates  adaptive,  interactive  simulations  of  actors  based  on  a  set  of 
conditions  stipulated  in  the  model  development  process.  For  example,  the  kinds  of  patterns 
extracted  through  agent-based  modeling  can  condition  the  parameterization  of  the  model  in 
subsequent  iterations.  In  this  way,  agent-based  modeling  not  only  allows  for  simulation  of  actors  in 
a  crowd  and  multiple  possible  configurations  of  interactions,  but  also  integrates  real  environmental 
factors  into  the  model  that  can  constrain  actors  in  their  subsequent  contacts  with  each  other.  In 
addition,  a  simulated  environment  created  with  agent-based  modeling  tools  can  incorporate 
physical  barriers  that  actors  must  negotiate  as  they  maneuver  through  the  model.  Finally,  agent- 
based  modeling  can  simulate  hierarchical  structure  and  visualize  various  alternative  scenarios  to 
explain  the  perceived  group  dynamics  in  near-real  time. 

7.2.  Visualization  Using  Self-Organizing  Maps 

In  the  field  of  context  awareness  and  artificial  intelligence,  a  self-organizing  map  (SOM)  is  an 
artificial  neural  network  that  allows  researchers  to  visualize  multidimensional  features  in  a  reduced 
dimensional  space  to  better  understand  the  similarities  between  attributes  (Hsu,  2006).  In  general, 
a  SOM  is  structured  as  a  topology  of  input  and  output  nodes,  also  referred  to  as  neurons  (Skupin  & 
Agarwal,  2008).  The  topology  can  be  either  trained  with  pre-defined  weight  vectors  or  untrained, 
with  each  input  vector  organized  using  a  random  weight  distribution.  After  the  SOM  has  clustered 
the  input  features  to  their  most  similar  nodes,  the  resulting  topology  becomes  available  for  further 
analysis  and  discovery. 

SOMs  provide  a  continuous  field  representation  of  multidimensional  data  reduced  to  lower 
dimensions  as  patterned  topologies  against  which  individual,  n-dimensional  features  can  be 
mapped  and  visualized  within  a  low-dimensional  space.  For  example,  Skupin  (2008)  created  a  SOM 
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to  visualize  how  a  hypothetical  researcher  might  discover  similarities  among  seemingly  disparate 
counties  across  the  geographic  space  from  Santa  Barbara,  CA,  to  New  Orleans,  LA,  using  only  the 
multidimensional  Census  attributes  of  income  distribution,  household  composition,  and  home 
ownership  of  each  county  visited.  Traditional  data  analysis  techniques  would  not  have  detected  the 
clustered  patterns  of  these  attributes  based  solely  upon  a  geographic  distribution.  This  attribute- 
driven  SOM  allowed  Skupin  to  infer  distinct  clustering  of  racial  distribution  and  home  ownership  as 
well  as  the  influences  of  household  composition.  In  a  second  experiment  described  in  the  same 
paper,  Skupin  again  used  Census  attribute  data  to  derive  the  socioeconomic  composition  of  New 
Orleans  neighborhoods  as  traversed  using  personal  transportation  and  public  transportation.  In 
this  SOM,  Skupin  analyzed  Census  block  data  (comprising  approximately  500  individuals  within  the 
metropolitan  area  of  New  Orleans)  to  reveal  the  distinct  neighborhood  compositions  within  the 
SOM's  topology. 


8.  Understanding  Within  Visualization 

Understanding  cultural  interactions  demands  a  complex  investigative  process,  regardless  of  how 
that  understanding  is  visualized.  While  the  technology  for  visualization,  based  upon  data  stores 
heretofore  unimaginable,  continues  to  improve  our  understanding  of  the  sociocultural  domain,  the 
principles  underlying  such  understanding  preoccupy  social  science  research.  The  promise  that  big 
data  holds  for  expanding  the  visualization  capabilities  of  predictive  modeling  is  tempered  by  the 
very  real  necessities  of  understanding  cultural  and  social  differences  in  the  present,  which  remains 
paramount  to  situational  awareness  and  therefore  a  strategic  priority.  The  proliferation  of 
visualization  techniques  and  methods  means  that  researchers  will  require  better  training  to 
appreciate  the  nuances  of  these  tools  and  to  constrain  the  data  inputs  in  order  to  obtain  predictive 
modeling  capabilities  vastly  different  from  those  currently  available. 

As  more  and  more  researchers  begin  to  tackle  the  very  real  challenges  of  using  vast  data  sets  to 
investigate  sociocultural  problems,  they  must  also  address  the  need  for  a  new  ontological  tenet  of 
scientific  investigation.  In  the  past,  datasets  that  were  single-sourced,  of  known  temporal  duration, 
and  collected  in  a  consistent,  documented  manner  held  the  researcher  to  a  standard  of  problem 
statement,  hypothesis  formulation,  hypothesis  testing,  acceptance,  or  rejection.  Today's  data 
challenge  is  how  to  rethink  data  processing  and  analysis  using  multi-source,  real-time,  multi¬ 
platform,  and  multi-formatted  data.  As  we  have  argued  in  this  chapter,  traditional  data  analysis 
and  testing  of  hypotheses  for  big  data  are  inadequate  to  address  the  multiplicity  of  dimensions, 
sources,  and  input  spaces  from  which  sociocultural  data  are  collected.  Agent-based  models  that 
perform  dynamic  integration  of  micro-scaled  actors  within  the  larger,  more  complex  and 
multidimensional  group  domain  or  SOMs  that  visualize  input  features  against  a  reduced 
dimensionality  topology  offer  two  examples  of  how  data  scientists  are  beginning  to  reformulate 
visualization  of  sociocultural  data  to  better  understand  the  underlying  dynamics  that  drive 
individuals  within  their  sociocultural  space. 
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4  Training  for  sociocultural  behavior  understanding  in 
operational  environments1 

Kyle  Behymer,  Julio  Mateo  &  Michael  McCloskey,  361  Interactive 
Allison  Abbe,  Synergist  Research  and  Consulting 


1.  Sociocultural  Behavior  Understanding 

From  the  intelligence  analyst  in  the  command  center  identifying  deviations  from  typical  patterns  of 
life  to  the  small  unit  leader  on  the  ground  negotiating  with  a  village  elder,  the  ability  to  understand 
sociocultural  behavior  is  vital  for  achieving  mission  success.  According  to  Schmorrow  (2011), 
accurate  analysis  of  sociocultural  behavior  requires  a  "thorough  perception  and  comprehension, 
grounded  in  social  and  behavioral  science,  of  the  sociocultural  features  and  dynamics  in  an 
operational  environment"  (p.  42). 

A  solid  understanding  of  sociocultural  behavior  enables  warfighters  to  recognize  subtle  details  that 
reveal  critical  information  about  cultural  and  social  norms  when  observing  interactions  in 
unfamiliar  cultural  settings.  For  example,  Figure  1  shows  a  group  of  Somalis  holding  a  meeting  at 
the  edge  of  their  village.  Though  many  individuals,  even  those  unfamiliar  with  Somali  culture,  may 
immediately  identify  the  leader  of  this  meeting  and  his  entourage  (the  group  standing),  they  might 
well  miss  the  more  subtle  cues  in  the  picture  that  would  allow  them  to  gain  a  more  in-depth 
understanding  of  the  cultural  environment.  For  example,  the  majority  of  the  men  not  part  of  the 
leadership  group  are  seated  in  the  shade,  while  the  women  and  children  are  seated  in  the  sun.  This 
observation  provides  information  about  the  likely  social  structure  of  this  society  and  the  role  that 
gender  plays. 


1  Approved  for  Public  Release;  Distribution  Unlimited.  Case  Number  14-2487 
This  work  was  supported  by  Department  of  Defense  Contract  W15P7T-13-C-F600 
Copyright  ©  2014  The  MITRE  Corporation. 
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Context  appreciation  and  an  understanding  of  normal  patterns  within  an  operating  environment 
are  crucial  for  making  sense  of  the  ambiguous  situations  that  warfighters  frequently  encounter. 
Such  understanding,  for  example,  enables  a  full  motion  video  (FMV)  analyst  tasked  with  providing 
convoy  overwatch  to  realize  that  a  single  shepherd  surrounded  by  a  flock  of  sixty  sheep  in  a 
position  that  could  provide  overwatch  of  the  route  is  probably  not  a  cause  for  concern;  however, 
three  'shepherds'  with  two  sheep  in  the  same  position  definitely  is.  Similarly,  a  member  of  a  route 
clearance  team  that  must  ensure  a  route  is  free  of  improvised  explosive  devices  (lEDs)  needs  a  solid 
understanding  of  social,  regional,  and  cultural  features  to  identify  and  make  sense  of  ambiguous 
cues.  For  example,  the  physical  environment  may  dictate  different  IED  emplacement  tactics: 
command  wire  lEDs  are  primarily  used  on  paved  roads  whereas  pressure  plate  lEDs  are  used  on 
unimproved  roads.  The  visual  indicators  of  these  types  of  lEDs  differ  and  can  be  very  hard  to 
detect,  as  they  may  include  very  common  items  (culverts,  yellow  jugs)  or  be  inherently  difficult  to 
see  (disturbed  earth).  Thus,  route  clearance  personnel  must  have  the  knowledge  to  determine 
whether  a  yellow  jug  by  the  roadside  is  being  used  as  a  delivery  vehicle  for  homemade  explosives 
or  to  transport  cooking  oil.  An  accurate  assessment  of  the  cultural  environment  is  critical  to  making 
sense  of  these  visual  cues. 

Unfortunately,  the  early  stages  of  recent  conflicts  in  Afghanistan  and  Iraq  demonstrated  a  clear 
capabilities  gap:  military  personnel  at  all  levels  received  insufficient  cultural  training,  troops  on  the 
ground  were  not  prepared  to  leverage  cultural  information  to  enable  mission  success,  and 
intelligence  analysts  did  not  integrate  cultural  analyses  into  their  products  (Connable,  2009).  As 
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one  soldier  stated,  "I  had  perfect  situational  awareness  [in  regard  to  things  such  as  the  location  of 
enemy  tanks].  What  I  lacked  was  cultural  awareness"  (McFate,  2005,  p.  43). 

To  address  these  gaps,  the  U.S.  Department  of  Defense  has  funded  government,  industry,  and 
university  laboratories  to  develop  effective  technology-based  training  for  understanding 
sociocultural  behavior.  This  chapter  discusses  these  state-of-the-art  training  technologies  and  how 
best  to  bring  them  into  operational  use.  We  first  summarize  training  technologies  already  in 
existence  or  currently  in  development.  Next,  we  discuss  the  gaps  in  existing  training  that  these 
technologies  can  help  bridge  as  well  as  the  challenges  associated  with  integrating  these  training 
technologies  into  the  operational  environment.  Finally,  we  provide  suggestions  on  overcoming 
these  implementation  challenges. 


2.  State-Of-The-Art  Training  Tools 

Given  the  scope  of  this  chapter,  we  do  not  attempt  to  provide  an  exhaustive  description  of  every 
available  or  emerging  technology  that  supports  training  in  sociocultural  understanding.  Instead,  we 
describe  existing  technologies  within  two  categories  based  on  their  primary  emphases: 
interpersonal  interaction  and  group-  or  population-level  dynamics.  We  briefly  describe  the  defining 
characteristics  of  each  category  and  provide  a  few  instances  of  technological  tools  under 
development  or  recently  developed. 

2.1.  Improving  Interpersonal  Interactions 

Tools  with  an  emphasis  on  interpersonal  interactions  use  text,  video,  or  a  virtual  environment  to 
present  users  with  a  scenario  involving  simulated  interaction  with  foreign  nationals.  Independent 
of  the  medium  used,  training  technologies  typically  ask  users  to  interpret  the  behavior  of  foreign 
nationals  within  a  situation  and/or  to  choose  culturally  appropriate  courses  of  action  at  different 
points  in  the  scenario.  Often,  the  scenarios  reinforce  lessons  taught  during  a  preceding  multimedia 
tutorial  for  the  specific  culture  in  which  the  scene  takes  place,  and  incorporate  multiple-choice 
questions  to  assess  the  user's  knowledge  (and  provide  feedback)  during  the  interaction. 

The  armed  services  often  implement  technological  tools  with  an  emphasis  on  interpersonal 
interactions  to  train  users  to  understand  and  interact  with  foreign  nationals  within  a  specific 
culture  (e.g.,  Afghanistan).  While  they  expect  some  skill  transfer  to  occur  across  cultures,  the 
developers  tailor  these  tools  for  specific  cultures  in  order  to  improve  the  sociocultural 
understanding  that  supports  face-to-face  interpersonal  interaction  with  members  of  that  culture. 
Thus,  they  often  integrate  common  acceptable  behaviors  and  cultural  taboos  (i.e.,  'do's  and 
don'ts')  of  the  specific  culture  into  their  lessons. 

Some  researchers  had  developed  structured  training  materials  and  exercises  that  addressed 
sociocultural  understanding  training  before  the  advent  of  sophisticated  computer  technologies 
(see  Bhawuk  &  Brislin,  2000,  for  a  review).  While  these  technologies  do  not  represent  the  state-of- 
the-art,  they  deserve  special  mention  as  the  predecessors  of  many  technologies  described  in  this 
section;  they  also  had  an  impact  on  the  training  techniques  currently  used  in  cultural  training 
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schools.  For  example,  culture  assimilators  (Fiedler,  Mitchell,  &  Triandis,  1971)  are  self- 
administered,  structured  training  programs  that  use  scenarios  to  expose  users  to  cross-cultural 
interactions  and  misunderstandings.  Users  answer  multiple-choice  questions  regarding  their 
understanding  of  the  incident  and  receive  tailored  feedback  (by  being  directed  to  different  pages) 
regarding  the  appropriateness  of  their  answer  and  the  rationale  behind  it.  Culture  assimilators 
have  undergone  extensive  evaluation  (Bhawuk  &  Brislin,  2000)  and  have  influenced  the 
development  of  both  training  curricula  and  technological  tools.  Most  of  the  more  technologically 
sophisticated  tools  discussed  below  follow  similar  formats,  but  leverage  video  and  virtual 
environments  in  an  attempt  to  create  higher  fidelity  and  more  engaging  scenarios.  They  also  have 
the  potential  advantage  of  providing  detailed  situational  context  that  text-based  scenarios  would 
lack  and  that  would  have  to  be  provided  by  a  human  instructor. 

Human-Actor-Based  Cultural  Scenarios 

These  technological  tools  feature  human  actors  playing  the  roles  of  U.S.  military  personnel  and 
foreign  nationals  in  a  cross-cultural  scenario.  While  they  present  details  about  the  scenario  in  a 
relatively  passive  way  (i.e.,  no  user  input),  movie-quality  videos  facilitate  'immersion'  in  the 
situation  and  engagement  during  the  sensemaking  and  decision  process.  Army  360  and  Visual 
Expeditionary  Skills  Training  (VEST)  fall  into  this  category.  Because  the  two  have  similar  structure 
and  format,  we  only  describe  Army  360,  which  resulted  from  a  2008-2009  partnership  between 
the  Army  and  InVism,  Inc.  Each  scenario  in  Army  360  is  preceded  by  a  'mission  briefing'  —  a 
multimedia  tutorial  with  information  relevant  to  the  upcoming  scenario,  and  a  pre-scenario  quiz  to 
estimate  the  user's  baseline  knowledge.  Since  these  elements  of  the  tool  are  specific  to  the  culture 
of  interest,  these  tutorials  impart  culture-specific  knowledge.  After  users  complete  the  pre¬ 
scenario  quiz,  Army  360  presents  them  with  movie-quality  reenactments  of  scenarios  in  the  foreign 
cultural  environment  of  interest.  At  different  points  during  the  scenario,  the  video  presentation 
pauses  and  users  are  presented  with  multiple-choice  questions  regarding  the  behavior  of  foreign 
nationals  or  the  course  of  action  the  user  should  take.  Typically,  users  are  expected  to  rely  on 
information  presented  during  the  pre-scenario  tutorial  and  to  demonstrate  learning  when 
answering  these  questions.  If  a  user  makes  an  inappropriate  choice  (e.g.,  asks  an  Iraqi  sheik  to  skip 
his  prayer  so  that  they  can  finish  their  discussion),  scenarios  present  negative  consequences 
associated  with  the  action  (e.g.,  the  Iraqi  sheik  angrily  walks  out  of  a  key  leader  engagement). 

Today,  U.S.  soldiers  and  airmen  can  access  Army  360  and  VEST  through  the  Army's  Training  and 
Doctrine  Command  (TRADOC)  Culture  Center  and  the  Air  Force  Culture  and  Language  Center 
(AFCLC),  respectively.  Versions  of  these  systems  are  available  for  a  range  of  countries,  including 
Iraq,  Afghanistan,  and  Somalia.  The  authors  are  unaware  of  any  formal  evaluation  conducted  to 
assess  the  effectiveness  of  these  tools  to  improve  the  sociocultural  understanding  of  warfighters  in 
operational  settings. 

Avatar-Based  Cultural  Scenarios 

These  technological  tools  rely  on  virtual  environments  and  avatars  (instead  of  human  actors)  to 
present  cultural  scenarios  similar  to  those  described  above.  Avatar-based  cultural  scenarios  offer 
the  distinct  advantage  of  greater  customizability  over  counterparts  that  use  human  actors,  since 
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the  reactions  of  the  avatars  can  be  better  tailored  to  the  user's  choices  and  actions  (Sagae,  Ho,  & 
Hobbs,  2012). 

The  Virtual  Cultural  Awareness  Trainer  (VCAT),  developed  by  Alelo  for  Joint  Knowledge  Online 
(JKO),  falls  into  this  category.  Like  the  tools  introduced  above,  VCAT  combines  multimedia  tutorials 
that  introduce  users  to  relevant  cultural  knowledge  (i.e.,  "knowledge-oriented  learning  activities/' 
Johnson,  Friedland,  Schrider,  Valente,  &  Sheridan,  2011)  and  scenarios  that  require  users  to  make 
decisions  about  the  meaning  of  the  foreign  national  behavior  and  preferred  courses  of  action  (i.e., 
"skill-oriented  role-playing  scenarios,"  Johnson  et  al.,  2011).  During  the  scenarios,  the  user  controls 
the  behaviors  and  communications  of  the  avatar  representing  the  U.S.  warfighter.  The  behavior  of 
the  avatars  representing  foreign  nationals  responds  to  the  user's  choices.  For  example,  "characters 
will  reciprocate  and  become  more  friendly  and  cooperative"  when  users  engage  in  appropriate 
behavior  and  build  rapport  with  foreign  nationals  (Johnson,  Friedland,  Watson,  &  Surface,  2012,  p. 
17).  VCAT  also  includes  supplementary  materials  regarding  language  (e.g.,  key  phrases  for  the 
region),  gestures,  and  advice  from  experienced  warfighters  in  the  form  of  interviews. 

While  the  authors  could  locate  no  specifics  on  evaluations,  Johnson  (2013)  reports  having 
"conducted  studies  to  evaluate  both  the  immediate  and  long-term  effectiveness  of  VCAT  training. 
Surveys  of  learners  and  their  supervisors,  conducted  after  the  learners  had  deployed  overseas, 
indicated  that  the  training  had  long-term  benefits"  (Johnson,  2013,  p.  2).  VCATs  for  Northern 
Africa,  the  horn  of  Africa,  South  America,  and  Afghanistan  are  available  through  JKO;  Alelo  reports 
it  is  also  developing  VCATs  for  Central  America,  the  Caribbean,  China,  and  Southeast  Asia  (Alelo, 
2013). 

2.2.  Understanding  Group-  or  Population-Level  Dynamics 

The  second  group  of  technology-based  tools  aims  at  developing  observation  skills  so  that  users  can 
better  understand  key  features  of  unfamiliar  cultural  environments.  While  these  tools  are  typically 
implemented  for  specific  (or  sets  of  specific)  cultures,  they  are  designed  to  train  culture-general 
skills  that  the  user  could  then  apply  to  any  novel  cultural  environment.  For  example,  these  tools 
emphasize  searching  for  patterns  of  behavior  in  the  environment  so  that  users  can  better 
understand  what  is  normal  in  that  setting  (i.e.,  'baseline'  or  identify  'patterns  of  life'),  and 
consequently  develop  the  ability  to  identify  deviations  from  the  baseline.  As  illustrated  in  the 
subcategories  below,  tools  within  this  category  vary  in  their  technological  sophistication  and 
degree  of  interactivity. 

Again,  structured  training  exercises  that  predated  the  development  of  more  sophisticated 
technological  tools  deserve  some  mention  here.  In  particular,  simulation  games,  such  as  BaFa  BaFa 
(Shirts,  2013),  have  certain  commonalities  with  technological  tools  under  this  category.  While  BaFa 
BaFa  has  had  greater  influence  on  training  schools  than  on  the  state-of-the-art  technological  tools 
presented  below,  this  structured  exercise  constitutes  an  example  of  an  early  attempt  to 
systematically  train  culture-general  observation  and  sensemaking  skills.  BaFa  BaFa  consists  of  a  1.5- 
hour  group  exercise  in  which  two  subgroups  are  assigned  to  two  fictitious  cultures  and  asked  to 
behave  according  to  a  set  of  cultural  rules.  One  member  of  each  subgroup  is  tasked  to  observe  the 


92  |  Training  for  sociocultural  behavior  understanding 


behavior  of  the  other  subgroup  and  learn  as  much  as  possible  without  asking  questions.  The 
observer  then  brings  this  information  to  his  or  her  subgroup,  the  subgroup  members  then 
collaboratively  attempt  to  make  sense  of  the  other  subgroup's  culture  and  formulate  hypotheses 
about  the  most  effective  way  to  interact  with  the  members  of  that  culture. 

Multimedia  Tutorials 

Multimedia  tools  typically  include  an  instructor-guided  lecture,  as  well  as  audiovisual 
demonstrations  of  the  concepts  being  explained.2  The  video  medium  facilitates  independent,  on- 
demand  learning,  as  well  as  the  use  of  realistic  examples.  In  all  of  the  instances  we  found,  these 
tutorials  were  used  to  complement  other  training  methods  (e.g.,  interaction-oriented  technologies, 
human-administered  training). 

As  an  example,  the  Marine  Corps  Warfighting  Laboratory  developed  Combat  Hunter  Computer- 
Based  Trainer  (CBT)  to  support  human-administered  training  that  facilitates  a  more  proactive 
mindset  in  operational  settings.  Specifically,  the  tool  aids  in  training  Marines  to  look  for  patterns  of 
life  so  that  they  can  more  effectively  identify  "what  is  here  that  should  not  be  here;  and  what 
should  be  here  that  is  not"  (Hilburn,  2007,  p.  60).  Drawing  lessons  from  rural  hunters,  individuals 
from  inner-city  backgrounds,  law  enforcement  officers,  and  experienced  warfighters,  the  Combat 
Hunter  CBT  includes  a  series  of  tasks  and  exercises  aimed  at  developing  users'  observation, 
profiling,  and  tracking  skills  in  operational  settings.  A  series  of  instructor-guided  tutorials  targets 
the  skills  (e.g.,  observation,  profiling)  to  be  learned,  as  well  as  the  use  of  tools  (e.g.,  binoculars)  to 
enhance  performance.  The  CBT  prepares  participants  for  the  Combat  Hunter  training  by  providing 
some  of  the  declarative  knowledge  beforehand  and  enabling  human  trainers  to  spend  more  time 
interacting  directly  with  users  to  train  them  in  the  desired  skills  (Schatz  &  Nicholson,  2012). 

The  Combat  Hunter  Fact  Sheet  (Office  of  Naval  Research,  2013,  p.  2)  reports  that  "empirical 
outcome  testing"  conducted  "with  control  and  experimental  groups  of  Marines  at  the  School  of 
Infantry  East  in  September  and  October  2010  ...  validated  the  utility,  engagement,  and  instructional 
efficacy  of  the  software."  The  Marine  Corps  currently  uses  the  Combat  Hunter  CBT  in  its  training 
program. 

Interactive  Data  Visualization 

Accessing  relevant  and  reliable  sociocultural  information  about  countries  of  interest  and  directly 
comparing  these  countries  along  key  dimensions  often  presents  a  challenge.  Interactive  data 
visualization  offers  a  centralized  medium  for  obtaining  comparable  data  on  key  dimensions  across 
countries.  Moving  away  from  an  exclusively  text-based  medium  can  assist  users  to  better  explore 
trends  within  a  country,  compare  high-level  cultural  variables  to  U.S.  norms,  and  generate 
expectations  for  upcoming  deployments. 


2  The  pre-interaction  tutorial  used  in  conjunction  with  the  interaction-oriented  technologies  described  in  the 
previous  section  would  also  fall  under  this  multimedia-tutorial  category.  However,  those  technologies 
emphasize  culture-specific  practices  and  knowledge  rather  than  culture-general  skills. 
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The  visualization  tool  in  the  Preparation  Module  of  CultureGear  falls  into  this  category.  361 
Interactive,  LLC,  is  developing  CultureGear  under  the  sponsorship  of  the  Office  of  Naval  Research 
(ONR).  The  visualization  component  aids  users  in  directly  comparing  critical  cultural  dimensions 
across  multiple  countries  and/or  within  a  single  country  across  time  (McCloskey,  Behymer,  & 
Mateo,  2012).  For  example,  Figure  2  shows  that  the  user  has  selected  'population  growth  rate' 
from  the  Social  Factors  tab  of  the  menu.  The  population  growth  rate  for  each  country  is 
represented  graphically  on  the  world  map  via  color  coding,  with  warm  colors  (red,  orange,  yellow) 
representing  positive  growth  rates  and  cool  colors  (blue,  green)  representing  negative  growth 
rates.  A  quick  glance  can  show  the  user  that  the  population  of  Europe  is  declining  while  the 
population  of  Africa  is  on  the  rise.  A  comparison  analysis  tool  enables  the  user  to  gain  more 
information  about  a  particular  country's  population  growth  rate  over  the  past  few  years  and  to 
view  trends  of  multiple  countries  side  by  side.  The  information  currently  used  in  CultureGear' s 
Preparation  Module  is  imported  from  the  online  Central  Intelligence  Agency  (CIA)  Factbook.  While 
reception  by  military  cultural  trainers  and  leadership  has  been  positive,  the  Preparation  Module 
has  not  yet  been  formally  evaluated  or  integrated  into  Armed  Forces  programs. 


Interactive  Tools  Using  Real-World  Imagery 

Interactive  tools  utilizing  real-world  images  to  train  the  perceptual/observational  skills  of 
warfighters  typically  ask  participants  to  identify  and  explain  aspects  of  a  scene  that  are  relevant  for 
sociocultural  understanding.  Both  the  Observations  Module  of  CultureGear  and  the  Difference- 
Spotting  Exercise  of  Combat  Hunter  fall  under  this  category.  The  Observations  Module  of 
CultureGear  consists  of  a  collection  of  geo-tagged  images  depicting  scenes  from  actual  U.S.  Army 
deployments  (many  images  include  both  U.S.  forces  and  foreign  nationals).  Users  are  asked  to  tag 
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the  critical  elements  or  aspects  within  each  scene  that  are  relevant  for  sociocultural  understanding 
and  to  explain  the  sociocultural  relevance  of  those  elements  or  aspects.  After  submitting  their 
responses,  users  compare  their  selections  and  rationales  with  those  of  a  consensus  of  culturally 
experienced  warfighters  and  cultural  experts  (see  Figure  3).  Users  also  complete  a  quiz  about  their 
understanding  of  the  scene.  Again,  they  can  check  whether  their  answer  to  each  question  matches 
the  experienced  warfighters'  responses  and  the  aspects  of  the  scene  that  experts  used  to  make 
their  assessments. 

Researchers  recently  assessed  the  effectiveness  of  CultureGear  in  improving  observation 
performance  among  junior  warfighters  (Mateo,  Behymer,  &  McCloskey,  2013).  Users  who  received 
CultureGear  training  showed  faster  and  more  accurate  performance  than  control  participants 
during  observation  tasks.  No  operational  forces  currently  use  CultureGear,  but  361  Interactive  has 
a  teaming  agreement  in  place  with  the  John  F.  Kennedy  Special  Warfare  Center  and  School 
(JFKSWCS)  to  support  its  integration  into  their  operational  training  curricula  over  the  next  three 
years. 


t  Child 
^  Electrical 

%  Power  lines 
%  Soldier 
%  Shop  wall 
^  Antenna 
Culvert 
^  Goat 
^  No  shoes 
^  Building  materials 
^  Produce 


Figure  3.  Screenshot  from  CultureGear' s  Observations  Module. 


The  Difference-Spotting  Exercise  of  Combat  Hunter  involves  the  presentation  of  two  seemingly 
identical  scenes  and  asks  users  to  identify  the  differences  between  them.  According  to  Hilburn 
(2007),  Marines  can  access  these  exercises  while  deployed  to  practice  their  observation  skills. 

Developers  are  also  building  interactive  tools  using  real-world  imagery  to  assist  intelligence, 
surveillance,  and  reconnaissance  (ISR)  analysts  in  understanding  unfamiliar  cultural  environments. 
The  type  of  imagery  used  for  the  ISR  domain  is  more  typically  a  bird's  eye  view  of  the  scene  than  a 
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ground-level  perspective.  However,  tools  such  as  the  Cognitive  Desktop  Analysis  Trainer  (C-DAT) 
use  a  procedure  similar  to  those  used  in  the  Observations  Module  of  Culture  Gear  and  the  Imagery 
Exercise  of  Combat  Hunter.  C-DAT  shows  FMV  analysts  both  simulated  and  real-world  FMV  and 
asks  them  to  identify  patterns  of  life  and  recognize  deviations  from  normal  activities  (e.g., 
unexpected  individual  behaviors)  in  real  time.  Users  also  answer  questions  that  probe  their 
sociocultural  understanding  of  the  cultural  scene.  For  example,  in  Figure  4  the  trainee  has  correctly 
identified  a  construction  crew  working  on  a  mosque  as  an  item  of  interest  and  now  must  select 
why  this  event  is  significant  from  the  list  on  the  right.  If  the  trainee  gives  an  incorrect  answer,  the 
tool  highlights  the  correct  answer  and  explains  the  rationale  behind  that  choice.  C-DAT  also  has  a 
game-like  scoring  system  to  increase  engagement  and  competitiveness  among  trainees.  As  C-DAT  is 
an  experimental  tool  currently  under  development,  its  effectiveness  has  not  been  formally 
evaluated,  nor  has  it  been  employed  in  operational  settings. 


Figure  4.  Screenshot  of  C-DAT  Training  System. 


Interactive  Tools  Using  Virtual  Environments 

Like  interaction-oriented  technological  tools,  interactive  tools  with  an  emphasis  on  understanding 
group-  or  population-level  dynamics  could  leverage  virtual-environment  technologies.  To  our 
knowledge,  no  current  technological  tools  use  virtual  environments  and  avatar-based  scenes  to 
improve  the  perceptual/observational  skills  of  users  beyond  interpersonal  interactions.  However, 
ONR  has  funded  the  PercepTS  project  to  create  such  a  tool.  Following  the  Combat  Hunter 
philosophy,  this  project  aims  at  developing  a  virtual  training  environment  ("Virtual  Ville")  in  which 
participants  can  complete  Combat-Hunter-like  training  (Schatz  &  Nicholson,  2012).  This  project  is 
expected  to  be  completed  in  the  2015-2016  timeframe. 
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3.  Science  And  Technology  Gaps 

3.1.  Lack  of  Training  Tools  for  Intelligence  Analysts 

"Although  the  Intelligence  Community  is  full  of  world-class  expertise  on  foreign  peoples,  places, 
and  organizations,  this  industry  rarely  isolates  and  illustrates  culture  as  a  factor  deserving  its  own 
sophisticated  and  thorough  treatment."  (Johnson  &  Berrett,  2011,  p.  2) 

Current  training  efforts  and  those  in  development  seem  to  focus  primarily  on  small  unit  leaders 
interacting  with  foreign  nationals  on  the  ground.  Many  existing  and  emerging  tools  are  designed  to 
improve  understanding  in  the  context  of  one-on-one  interactions  as  opposed  to  understanding 
group  or  population-level  dynamics.  By  contrast,  almost  no  training  tools  exist  for  the  intelligence 
analysis  and  strategic  planning  levels,  even  though  dramatic  changes  in  sensor  technologies  and 
the  nature  of  warfare  over  the  past  several  decades  have  made  it  increasingly  important  for  ISR 
analysts  to  have  sociocultural  understanding  to  successfully  detect  and  interpret  critical  events. 

During  the  Cold  War  era,  ISR  analyst  tasking  was  relatively  simple,  perhaps  involving  monitoring  of 
an  adversary's  physical  assets,  including  tanks,  missiles,  and  planes  (Bryant,  Johnson,  Kent,  Nowak, 
&  Rogers,  2008).  Today's  ISR  analyst,  however,  may  be  tasked  to  detect,  observe,  interpret,  and 
predict  patterns  of  life  activities— for  example,  to  identify  networks  associated  with  terrorist  cells — 
while  simultaneously  assessing  the  cultural  and  contextual  aspects  of  the  surrounding  regions.  The 
same  visual  stimulus— a  man  digging  near  a  road— might  represent  a  critical  event  that  the  analyst 
must  not  only  detect  but  also  understand  within  its  context:  Is  that  individual  a  farmer  digging  a 
trough  for  water  or  an  insurgent  emplacing  an  IED? 

Despite  the  challenges  facing  intelligence  analysts  and  the  limitations  of  existing  training,  little 
research  has  focused  on  designing  effective  training  to  develop  analytic  skills.  Existing  pre-mission 
training  often  focuses  on  basic  intelligence  principles  and  analysts  only  gain  an  understanding  of 
sociocultural  behavior  through  on-the-job  training.  This  often  places  analysts  in  difficult  situations. 
Rather  than  beginning  their  training  with  a  relatively  simple  task  (e.g.,  calling  out  vehicular  traffic  in 
a  rural  setting),  many  analysts  must  learn  by  confronting  a  very  complex  real-world  mission 
scenario  that  involves  monitoring  multiple  named  areas  of  interest  while  trying  to  fulfill  multiple 
customer  intelligence  requests. 

3.2.  Valid  Content  Often  Missing 

Tool  designers  often  place  too  much  focus  on  developing  the  training  technology  as  opposed  to 
developing  the  content  to  populate  it.  To  be  useful,  training  in  sociocultural  understanding  must 
rest  on  a  solid  understanding  of  real-world  constraints.  One  useful  method  for  developing  training 
content  is  the  decision  requirements  table  (DRT),  a  technique  developed  by  Klein  Associates,  Inc., 
which  captures  the  critical  cues  that  experienced  analysts  use  when  making  a  cognitively 
demanding  decision  (Phillips,  McDermott,  Thordsen,  McCloskey,  &  Klein,  1998).  The  DRT  can  also 
capture  why  a  decision  is  difficult,  the  information  sources  that  experienced  analysts  consult  to 
inform  their  decision,  the  common  errors  novices  make  when  reaching  a  decision,  the  strategies 
used  by  experienced  analysts,  and  the  sources  of  uncertainty. 
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Table  1.  Sample ,  partial  DRT 
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Table  1  shows  a  sample  partial  DRT  based  on  a  small  set  of  cognitive  task  analysis  (CTA)  interviews 
conducted  with  the  leaders  of  small  Marine  Corps  units.  The  decision  requirement  is  identified  at 
the  top  of  the  table  and  the  first  column  describes  some  specific  reasons  that  can  make  this 
decision  particularly  challenging.  In  this  case,  the  reasons  why  interviewees  found  economic 
analysis  challenging  in  the  context  of  specific  situations  included  that  informal  economic  activities 
are  hard  to  recognize  and  are  unregulated,  and  that  individuals  are  often  reluctant  to  discuss  them. 
The  second  column  describes  the  cues  that  can  help  analysts  evaluate  the  economy,  such  as  overt 
signs  of  black  market  activity.  The  third  column  describes  information  sources  that  individuals  may 
consult  when  making  the  decision  or  knowledge  they  need  to  have  (e.g.,  rules  and  laws  regarding 
what  activities  are  legal  in  a  region).  The  fourth  column  identifies  common  errors  that  novices 
might  make  (e.g.,  assuming  that  only  formal,  regulated,  and  legal  activities  are  part  of  the 
economy).  The  fifth  column  lists  strategies  that  experts  may  use  to  support  their  decision  making, 
such  as  active  monitoring  for  illegal/unregulated  activities.  The  final  column  identifies  common 
sources  of  uncertainty  that  may  increase  the  difficulty  of  making  an  assessment  or  decision. 

DRTs  can  be  used  in  several  ways  to  direct  scenario  development.  They  serve  as  sources  of 
information  about  how  to  make  a  scenario  more  or  less  challenging.  They  also  indicate  means  for 
evaluating  the  trainees'  decisions  during  a  scenario  by  outlining  common  errors  and  successful 
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strategies  and  by  denoting  the  knowledge  that  should  be  presented  or  available  to  trainees  within 
or  before  the  scenario  as  well  as  the  resources  trainees  should  be  able  to  access.  They  also  contain 
the  critical  perceptual  cues  that  should  be  represented  within  cognitively  authentic  scenarios. 

3.3.  Limitations  of  Frameworks  as  a  Tool  for  Understanding 

While  frameworks  are  typically  seen  as  useful  tools  to  aid  understanding  of  new  environments, 
they  may  lead  to  oversimplification  of  sociocultural  complexity.  Additionally,  training  tools  based 
on  these  frameworks  often  assume  that  warfighters  can  ascertain  ground  truth.  For  example,  one 
of  the  tools  we  examined  presents  trainees  with  the  following  information:  "In  Iraq,  Shia  Muslims 
typically  wear  black-and-white  headdresses  while  Sunni  Muslims  wear  red-and-white 
headdresses."  Trainees  are  then  shown  a  video  of  an  Iraqi  wearing  a  black-and-white  headdress 
and  asked  to  identify  whether  he  is  a  Shia  Muslim  or  a  Sunni  Muslim.  Compare  this  situation  to  the 
following  training  scenario,  derived  from  a  real-world  incident  described  to  us  during  a  CTA 
interview: 

It  is  now  the  third  month  of  your  mission  and  you  have  established  a  friendly,  yet  fragile, 
relationship  with  a  local  village  elder.  One  day  during  a  standard  patrol  of  the  village,  your 
interpreter  begins  a  conversation  with  a  trusted  "source"  on  the  street.  The  source  tells  your 
interpreter  that  one  of  the  top  insurgents  in  the  country  is  holing  up  in  a  house  on  the  far  side 
of  the  village.  To  capture  this  insurgent  would  put  a  major  dent  in  the  insurgents'  networks. 
You  head  toward  the  house  and  on  the  way  you  come  across  the  village  elder.  You  inform  him 
of  this  intelligence  and  he  immediately  gets  visibly  upset.  In  that  house,  he  tells  you,  is  one  of 
the  most  respected  families  in  the  village,  and  they  are  close  personal  friends  of  the  elder.  He 
assures  you  that  your  intelligence  is  wrong  and  that  there  is  no  way  they  could  be  holding  this 
insurgent.  If  you  raid  the  house,  he  says,  the  entire  village  will  be  very  angry,  including  himself 
and  his  councilmen,  who  have  just  arrived  and  who  seem  as  irritated  as  he  is. 

In  this  far  more  complex  situation  the  warfighter  cannot  establish  ground  truth  before  taking 
action.  If  the  warfighter  opts  to  raid  the  house  and  the  insurgent  is  not  there  it  could  have  tragic 
consequences  for  U.S.  forces'  relationship  with  the  village  elder  and  the  village.  If  the  warfighter 
decides  to  trust  the  elder  and  it  turns  out  that  the  insurgent  is  in  the  house,  not  capturing  him  may 
have  tragic  consequences  for  U.S.  forces. 

To  prepare  warfighters  for  this  type  of  ambiguous  situation,  cultural  training  must  teach  trainees 
"how  to  think"  rather  than  "what  to  think."  Training  limited  to  "this  category  of  people  wear  this 
kind  of  clothing"  offers  little  help  for  understanding  sociocultural  behavior;  instead,  training  should 
focus  on  developing  the  skills  necessary  to  make  informed  decisions  in  complex  cross-cultural 
situations  (Salmoni  &  Holmes-Eber,  2008). 

3.4.  Lack  of  Easily  Updatable  Training  Tools 

During  the  past  ten  years  many  developers  of  cultural  training  tools,  whether  cultural-general, 
region-specific,  or  language  tools,  had  a  specific  goal:  supporting  warfighters  deployed  to 
Afghanistan  and  Iraq.  To  this  end,  the  majority  of  training  tools  we  have  reviewed  use  training 
scenarios  set  in  one  of  these  two  countries.  However,  the  U.S.  military  has  already  withdrawn  from 


Training  for  sociocultural  behavior  understanding  |  99 


Iraq  and  is  planning  to  withdraw  from  Afghanistan.  Thus,  end  users  may  not  have  a  positive  view  of 
training  whose  content  focuses  solely  on  these  countries.  As  one  trainer  told  us,  "We're  out  of  Iraq, 

I  don't  care  about  Iraq." 

As  the  military  cannot  predict  future  deployment  locations  with  complete  accuracy,  warfighters 
must  be  prepared  to  understand  sociocultural  behavior  whether  they  will  assist  in  disaster  relief 
efforts  in  Haiti,  provide  intelligence  to  North  Atlantic  Treaty  Organization  (NATO)  forces  operating 
in  Libya,  or  intervene  in  failed  nation-states  such  as  Somalia.  Effective  training  tools  should  include 
scenarios  in  multiple  cultural  environments  with  the  goal  of  imparting  skills  that  can  be  effective  in 
any  cultural  environment.  Even  region-specific  training  tools  should  include  mechanisms  that  allow 
the  trainers  who  use  them  to  easily  update  existing  content  or  generate  new  content  appropriate 
to  the  next  conflict.  Both  cultural  trainers  and  trainees  may  soon  view  a  training  tool  that  cannot 
be  easily  updated  as  obsolete. 

3.5.  Lack  of  Empirical  Validation 

Few  of  the  technological  tools  developed  to  improve  warfighters'  sociocultural  understanding  in 
operational  settings  have  undergone  significant  empirical  validation.  This  gap  applies  to  both 
didactic  methodology  and  content. 

Didactic  Methodology 

The  tools  described  in  the  previous  section  use  various  means  to  promote  trainee  learning.  For 
example,  Combat  Hunter  CBT,  Army  360,  and  VCAT  include  a  tutorial  as  part  of  their  didactic 
methodology,  whereas  CultureGear  does  not.  Most  (e.g.,  Army  360,  VEST,  CultureGear's 
Observations  Module)  include  quizzes  as  part  of  their  testing/feedback  mechanism,  but  others  do 
not  (e.g.,  Combat  Hunter  CBT,  CultureGear's  Preparation  Module). 

Tools  provide  performance  feedback  along  a  continuum  from  direct/explicit  to  indirect/implicit.  For 
instance,  some  of  the  tools  with  quizzes  (e.g.,  CultureGear's  Observations  Module,  C-DAT)  give 
explicit  'right/wrong'  feedback  to  trainees  and  explain  the  rationale;  other  tools  (e.g.,  Army  360, 
VEST)  provide  'right/wrong'  feedback  by  allowing  learners  to  experience  the  consequences  of  their 
choice;  and  avatar-based  tools  include  more  implicit  feedback  (e.g.,  increasingly  subtle  changes  in 
avatar  behavior  when  the  user  engages  in  appropriate  behaviors).  For  the  most  part,  these  tools  do 
not  incorporate  any  explicit  mechanism  to  encourage  practice  of  appropriate  behavior  among 
trainees.  However,  game-based  tools  such  as  C-DAT  do  convert  performance  into  a  score  that 
trainees  attempt  to  maximize,  thus  encouraging  trainees  to  supply  appropriate  responses  and 
compete  successfully  with  peers. 

Independent  of  the  content  taught,  a  tool  must  demonstrate  its  ability  to  effect  change  in  the 
trainee.  Quantitative  evaluations  of  didactic  methodologies  used  in  tools  to  enhance  sociocultural 
understanding  are  rare  (see  Mateo  et  al.,  2013,  for  one  exception).  Even  rarer  are  comparisons 
across  different  didactic  methodologies  to  determine  the  most  effective  feedback  mechanisms  and 
guide  the  development  of  future  tools  to  enhance  sociocultural  understanding. 
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Content 

While  ensuring  that  a  tool  can  improve  trainee  knowledge  is  an  important  first  step,  military 
organizations  must  also  know  that  the  resulting  change  will  ultimately  have  a  positive  impact  on 
sociocultural  understanding  and  performance  in  operational  settings.  Imagine  for  a  moment  that 
you  have  two  weeks  to  provide  cultural  training  to  a  class  of  20  warfighters.  Researchers  from  a 
government,  industry,  or  academic  laboratory  present  you  with  a  cultural  training  tool  that,  on  the 
surface,  appears  to  fit  well  with  your  training  curricula.  Before  you  decide  to  spend  your  limited 
amount  of  time  and  departmental  resources  on  incorporating  this  tool  into  your  curricula  you  want 
some  assurance.  How  can  you  know  that  this  tool  will  actually  improve  trainee  performance?  In 
other  words,  how  valid  is  the  content  of  the  training  tool? 

Accurate  measurement  of  mission  effectiveness,  especially  as  it  relates  to  the  role  played  by 
sociocultural  understanding,  is  critical  to  validate  training  technologies.  Existing  training  rarely  has 
good  assessment  techniques  to  evaluate  the  usefulness  of  the  content  as  it  relates  to  performance 
in  operational  settings  (mission  effectiveness).  Unfortunately,  little  empirical  evidence  exists  to 
demonstrate  that  cultural  training  improves  the  mission  effectiveness  of  warfighters  (DeCamp  et 
al.,  2012).  A  key  reason  for  this  lack  of  empirical  content  validation  lies  in  the  lack  of  adequate 
methods  for  measuring  mission  effectiveness  (DeCamp,  Meadows,  Costa,  Williams,  Bornmann,  & 
Overton,  2012).  The  Department  of  Defense  would  benefit  from  methods  that  can  measure  and 
assess  individuals'  understanding  of  sociocultural  behavior  in  an  operational  environment. 

4.  Barriers  To  Adoption 

In  the  authors'  experience,  several  barriers  currently  inhibit  the  adoption  of  technologies  to 
support  cultural  training.  All  these  challenges  may  have  a  common  feature:  the  inherent 
complexity  and  uncertainty  of  the  underlying  constructs  that  such  training  addresses.  Typical 
training  efforts  have  clear  and  certain  objectives:  they  train  warfighters  to  be  better  marksmen, 
more  knowledgeable  tacticians,  or  clearer  communicators.  But  the  competencies  that  underlie  the 
inherently  complex  construct  referred  to  as  sociocultural  understanding  are  still  under  debate 
within  the  research  and  operational  communities.  Should  training  focus  on  affective  attributes 
(e.g.,  changing  perceptions,  biases,  and  attitudes),  behavioral  skills  (e.g.,  developing  better 
persuasive  techniques),  or  cognitive  skills  (e.g.,  observing  and  interpreting  intercultural  dynamics  in 
complex  environments)?  The  answer  seems  to  be  a  combination  of  all  three,  and  likely  depends 
upon  the  mission  sets  of  the  specific  recipient  group. 

Cross-cultural  skills  remain  a  nascent  concept  within  the  military  training  and  research 
communities,  and  are  associated  with  a  wide  range  of  theoretical  models  and  guiding  frameworks. 
Each  military  branch  (and  within-branch  training  group)  must  choose  the  most  appropriate 
foundations  for  its  purposes.  As  a  result,  discord  abounds  over  the  identification  of  the  "one  best" 
cultural  competence  model  to  promote,  or  the  single  set  of  baseline  skills  to  train. 

4.1.  Training  Differences  among  Military  Branches 

"The  cultural  awareness  training  landscape  is  diverse,  with  an  array  of  textures,  colors,  and  hues. 
Proverbially  speaking,  what  is  missing  from  the  picture  is  the  frame"  (Alrich,  2008,  p.  2). 
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In  a  study  that  sought  to  describe  cultural  training  programs  within  the  U.S.  military,  Alrich  (2008) 
discovered  a  diverse  and  varied  cultural  training  landscape  that  contained  a  large  number  of 
programs  with  differing  theoretical  foundations,  user  populations,  and  missions.  Each  training 
center  has  its  own  preferred  theoretical  frameworks  and  training  objectives  for  sociocultural 
understanding. 

For  example,  the  U.S.  Army's  JFKSWCS,  which  provides  training  for  Army  Civil  Affairs,  Psychological 
Operations,  and  Special  Forces  soldiers,  uses  the  PMESII-PT  (Political,  Military,  Economic,  Social, 
Infrastructure,  Information,  Physical  Environment,  and  Time)  model  to  train  soldiers  in 
understanding  sociocultural  behavior  in  operational  environments.  JFKSWCS  offers  two  courses 
that  focus  on  sociocultural  understanding:  Foundations  of  Cross-cultural  Competence  (FC3)  and 
Regional  Analysis.  In  the  FC3  course,  students  gain  understanding  about  the  components  of  culture 
by  examining  their  own  cultures,  learning  how  to  manage  their  perceptions  about  other  cultures, 
and  investigating  how  other  cultures  view  U.S.  culture.  Specific  learning  objectives  (McCloskey, 
Behymer,  Papautsky,  Ross,  &  Abbe,  2010)  include:  identify  the  components  of  cross-cultural 
competence  that  promote  the  development  of  cross-cultural  competence  and  facilitate  mission 
success;  demonstrate  an  openness  to  alternative  explanations  and  ideas;  recognize  that  cultures 
differ  in  significant  and  meaningful  ways  and  that  these  differences  influence  behavior;  formulate 
accurate  cross-cultural  understandings  and  assessment  of  situational  dynamics,  the  perspectives  of 
others,  and  the  impact  of  cultural  actions  on  the  broader  mission  as  well  as  secondary  and  tertiary 
effects;  and  demonstrate  the  ability  to  consistently  present  oneself  in  a  manner  that  promotes 
positive  short-term  and  long-term  relationships.  The  Regional  Analysis  course  teaches  students  a 
systems  approach,  with  the  desired  outcome  being  an  increase  in  their  cultural  competencies  and 
an  understanding  of  countries  or  regions  based  on  the  application  of  the  PMESII-PT  operational 
variables. 

By  contrast,  the  U.S.  Marine  Corps  Center  for  Advanced  Operational  Culture  Learning  (CAOCL)—  the 
central  Marine  Corps  agency  for  operational  culture  and  language  familiarization— prefers  to  use 
the  Five  Dimensions  of  Operational  Culture  model  (Salmoni  and  Holmes-Eber,  2008),  which  states 
that  physical  environment,  economy,  social  structure,  political  structure,  and  belief  systems  are 
especially  critical  to  understanding  sociocultural  behavior.  CAOCL's  principal  educational  activity 
consists  of  managing  the  Regional,  Culture,  and  Language  Familiarization  (RCLF)  Program:  a  career- 
length  program  designed  to  instill,  develop,  and  sustain  basic  language,  region,  and  culture 
capabilities  in  career  Marines.  The  RCLF  Program  uses  the  five  dimensions  of  culture  to  provide  a 
cross-cultural  competence  foundation  through  culture-general  training  (helping  Marines  determine 
what  they  need  to  know  and  how  they  can  build  that  knowledge),  culture-specific  training  (showing 
the  unique  characteristics  of  the  operating  area  organized  according  to  the  five  dimensions  of 
culture),  and  language  familiarization  (memorizing  key  phrases  that  enable  mission 
accomplishment). 

These  differences  are  not  only  found  between  Military  Services:  training  centers  within  the  same 
Service  often  have  different  training  theoretical  foundations  and  training  objectives  as  well.  For 
example,  in  contrast  to  the  approach  used  at  JFKSWCS,  the  Reserve  Officers'  Training  Corps  (ROTC) 
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Cultural  Understanding  and  Language  Proficiency  (CULP)  focuses  on  preparing  participants  in  the 
ROTC  Culture  and  Language  Immersion  Internship  Program.  During  that  three-week  program  the 
cadets  interact  with  the  local  populace  (in  countries  such  as  China,  Egypt,  Ghana,  Mongolia, 
Morocco,  Russia,  Senegal,  Taiwan,  and  Tanzania)  to  perform  a  variety  of  humanitarian  missions. 
The  cadets  strive  to  achieve  cultural  competencies  based  on  the  Army  Culture  and  Foreign 
Language  Strategy  Objectives  (2009).  A  sample  objective  is: 

Demonstrates  an  awareness  of  own  cultural  assumptions,  values,  and  biases  and  understands 
how  the  U.S.  is  viewed  by  members  of  other  cultures;  applies  perspective  taking  skills  to 
detect,  analyze,  and  consider  the  point  of  view  of  others  and  recognizes  how  own  actions  may 
be  interpreted. 

These  differences  in  training  methods  and  objectives  create  several  challenges  for  developers  of 
training  technologies  who  hope  to  create  tools  that  are  easily  generalizable  across  branches  of  the 
Armed  Services.  Despite  the  similarities  between  the  PMESII-PT  model  and  the  Salmoni  &  Holmes- 
Eber  model,  differences  in  terminology  can  alienate  both  trainers  and  trainees.  Even  simple  visual 
differences  can  displease  potential  end  users.  For  example,  during  a  presentation  of  an  early 
version  of  the  CultureGear  tool  at  a  cultural  training  center,  a  trainer  expressed  irritation  that  a 
camouflage  pattern  on  the  main  screen  came  from  a  different  branch  of  the  military.  While 
technology  developers  may  have  little  difficulty  adapting  a  camouflage  pattern  for  each  service, 
trying  to  incorporate  a  framework  with  different  terminology  throughout  an  entire  training  tool 
(e.g.,  replacing  PMESII-PT  with  the  Five  Dimensions  of  Operational  Culture)  presents  significant 
difficulty.  Developers  hoping  to  transition  their  state-of-the-art  training  tools  into  operational  use 
must  successfully  navigate  this  complex  cultural  training  landscape. 

4.2.  Lack  of  Communication  Between  Trainers  and  Technology  Developers 

When  the  need  for  cross-cultural  competence  training  became  clear,  the  U.S.  Armed  Forces 
created  training  centers  and  tasked  them  with  incorporating  culture  within  their  existing  curricula 
or  developing  techniques  to  train  cross-cultural  warfighters  (Alrich,  2008).  At  the  same  time,  the 
services  directed  researchers  and  technologists  from  government,  industry,  and  university 
laboratories  to  develop  technological  solutions  to  support  training  in  cross-cultural  competence. 
Although  these  two  lines  of  research  and  development  have  evolved  in  parallel  over  the  past  ten 
years,  they  have  not  communicated  effectively.  Technology  developers  have  often  focused  their 
time  and  energy  on  technological  advances  (such  as  increasing  the  physical  fidelity  of  virtual 
worlds)  as  opposed  to  determining  how  the  end  product  could  best  support  the  most  likely  end 
users:  trainers  and  trainees  within  military  cultural  training  programs. 

If  technological  tools  are  to  be  systematically  integrated  and  utilized  into  military  training  centers, 
these  technologies  must  rest  on  a  solid  understanding  of  the  training  truly  needed,  the  training 
provided  in  these  centers,  and  the  standards  and  regulations  that  the  training  must  meet,  and  must 
complement  and  enhance  this  training.  Only  then  will  the  resulting  training  programs 
synergistically  leverage  the  strengths  of  training  and  technological  approaches  to  support  the 
development  of  culturally  competent,  cognitively  ready  U.S.  military  personnel. 
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Additionally,  it  is  often  difficult  (if  not  impossible)  for  training  developers  to  identify  individuals 
within  existing  training  centers  who  will  remain  in  position  long  enough  to  champion  a  new  training 
tool  throughout  its  developmental  cycle.  In  many  cases  the  key  leader  with  whom  technology 
developers  partnered  to  ensure  successful  integration  is  reassigned  to  a  different  position  before 
the  tool  can  be  deployed.  In  these  cases,  training  developers  must  start  from  scratch  with  the 
newly  appointed  leader,  who  may  have  substantially  different  training  goals  and  priorities  from  his 
or  her  predecessor. 

4.3.  Time  Limitations 

Time  presents  one  of  the  biggest  barriers  to  implementing  new  training  technologies.  Warfighter 
training  schedules  are  already  very  full  (Abbe  &  Bortnick,  2010),  and  existing  cultural  training 
programs  must  essentially  compete  with  other  training  programs  that  warfighters  must  complete 
(Alrich,  2008).  While  each  branch  of  the  military  has  implemented  a  strategy  for  incorporating 
cultural  training  into  its  training  curricula  (Abbe  &  Gouge,  2012),  the  cultural  trainers  we  have 
interviewed  still  struggle  to  fit  the  material  they  consider  essential  into  the  limited  amount  of  time 
they  have  to  teach  it.  Often,  the  training  schedule  simply  does  not  leave  enough  time  for  cultural 
trainers  to  incorporate  new  technologies  into  their  lesson  plans  if  these  technologies  do  not  fit 
seamlessly  into  the  trainer's  existing  curricula.  If  a  new  training  technology  requires  an  additional 
time  commitment  from  the  warfighter,  or  would  take  valuable  time  from  the  already  limited 
number  of  training  hours  cultural  trainers  have  with  students,  the  operational  community  will  not 
accept  it. 

Furthermore,  cultural  trainers  often  have  only  limited  time  available  to  work  with  technology 
developers.  They  may  therefore  hesitate  to  volunteer  their  time  unless  they  both  see  the  value  of 
the  technology  and  can  demonstrate  the  value  of  that  technology  to  their  leadership. 

4.4.  Severely  Limited  Training  Budgets 

In  addition  to  restrictions  on  available  time  for  the  insertion  of  new  training  technologies,  the 
military  faces  increasing  limits  on  the  monetary  resources  available  for  training.  While  most 
organizations  apparently  consider  a  minimum  of  cultural  training  to  be  critical,  that  minimum 
typically  focuses  on  basic  knowledge  of  language  and  customs,  with  skills  in  critical  assessment, 
awareness,  and  interaction  viewed  as  nonessential.  Warfighters  must  know  to  apply  tactics, 
function  as  part  of  a  cohesive  team,  and  be  masters  of  their  weapons  before  the  services  can 
implement  other  training  initiatives.  Discretionary  training  that  involves  advanced  technologies 
often  comes  with  a  hefty  price  tag  that  can  be  inherently  inhibiting  in  military  settings  where 
decision  makers  are  already  struggling  with  continually  shrinking  training  budgets.  To  achieve 
acceptance  within  military  training  communities,  developed  technologies  must  demonstrate  that 
they  meet  a  recognized  critical  need,  can  contribute  to  reducing  overall  training  cost,  and  can 
accommodate  training  time  restraints. 

4.5.  Inconsistent  Leadership  Recognition  of  Mission  Relevance 

Across  the  Services  and  levels  of  command,  some  individuals  still  either  refuse,  or  are  unable,  to 
recognize  the  importance  of  cross-cultural  competence  for  warfighters.  A  traditional  mentality  of 
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"we  are  trained  to  fight,  not  to  befriend"  is  so  ingrained  in  some  individuals  that  they  see  cultural 
abilities  as  a  weakness  and  a  focus  on  cultural  awareness  as  a  way  to  undermine  the  military.  This 
mindset  has  become  less  prevalent  as  military  personnel  continue  to  gain  appreciation  of  the  link 
between  cultural  competence  in  the  operational  environment  and  mission  success,  but  it  still 
lingers.  Until  all  service  members  recognize  the  importance  of  cultural  skills  to  achieving  mission 
success,  resistance  to  training  technologies  that  claim  to  promote  it  will  remain. 

4.6.  Communication  Within  and  Among  Funding  Agencies  and  Military  Branches 

Currently,  communication  among  the  different  military  branches  with  regard  to  training  in 
sociocultural  understanding  can  best  be  described  as  ineffective.  Alrich  (2008,  p.  2)  states  that 
communication  between  different  training  programs,  both  between  and  within  branches,  is  limited 
by  "parochialism  and  defensiveness"  as  each  training  program  has  different  priorities,  objectives, 
and  terminology.  This  lack  of  communication  may  lead  to  duplication  of  effort,  and  may  also  inhibit 
the  cross-fertilization  of  state-of-the-art  training  methodologies  across  the  Services. 

4.7.  Failing  to  Learn  from  Previous  Experiences 

We  did  not  understand  what  was  going  on  in  ...  We  were  in  a  foreign  land  among  people  of  a 
different  culture  and  mind-set.  It  was  not  possible  to  translate  our  objectives  and  strategies 
into  actions  taken  by  [them]  ...  The  information  sent  across  the  cultural  divide  was  not  the 
information  received.  There  was  a  disconnect.  One  thing  was  said  and  another  thing  was 
heard.  The  truth  is,  no  one  knew  the  truth.  Meaning,  intent,  and  truth  were  lost  in  translation 
(Dockery,  2003,  p.  93). 

The  above  quotation  could  very  well  have  described  warfighters'  experiences  in  Iraq  and 
Afghanistan,  but  it  actually  came  from  a  soldier  assigned  to  an  Army  of  the  Republic  of  Vietnam 
combat  unit  during  the  Vietnam  War.  This  illustrates  that  the  lack  of  adequate  cross-cultural 
training  for  warfighters  long  antedates  operations  in  Iraq  and  Afghanistan.  In  fact,  during  the 
Vietnam  War  the  U.S.  Army  discovered  that  its  soldiers  lacked  the  fundamental  cross-cultural  skills 
necessary  to  interact  successfully  with  locals.  As  a  result,  the  Army  initiated  many  training 
programs  to  address  this  capabilities  gap  (Laughrey,  2008).  Unfortunately,  when  the  conflict  ended 
the  Army  returned  to  a  focus  on  training  in  conventional  warfare  (in  response  to  possible  moves  by 
the  Soviet  Union)  rather  than  continuing  to  focus  on  skills  (such  as  sociocultural  understanding) 
essential  for  counter-insurgency  and  nation-building  (Laughrey,  2008).  With  this  change  in  focus, 
the  lessons  learned  in  Vietnam  were  lost. 

Recently  adopted  policies  that  incorporate  culture  into  doctrine  provide  hope  that  the  military  will 
not  forget  the  lessons  relearned  in  Iraq  and  Afghanistan  (Abbe  &  Gouge,  2012).  However,  the 
military  cannot  rely  solely  on  these  doctrinal  changes.  Training  must  capture  the  subject  matter 
expertise  obtained  by  soldiers  who  have  returned  from  deployments  in  Iraq  and  Afghanistan  so 
that  future  trainees  can  learn  from  their  experiences. 
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5.  Looking  Forward:  Getting  Technological  Tools  Into  Operational  Use 

"...  effective  sociocultural  education  requires  a  significant  investment  in  resources  and  there  is  no 
'quick  fix'  or  'shake  and  bake'  solution"  (Laughrey,  2008,  p.  16). 

As  the  previous  section  indicates,  many  potential  obstacles  prevent  the  Armed  Services  from 
implementing  state-of-the-art  training  technologies  within  the  operational  community.  The  military 
has  a  myriad  of  cultural  training  centers,  each  with  its  own  mission  sets,  priorities,  and 
terminology.  Trainers  within  these  centers  often  hesitate  to  adopt  training  tools  that  might  cut  into 
the  limited  time  and  resources  they  have  for  training  (and  that  may  not  actually  improve  trainee 
performance).  Developers  of  training  technology,  on  the  other  hand,  often  spend  too  much  time 
concentrating  on  the  technology  itself  rather  than  on  determining  how  their  tools  can  best  assist 
cultural  trainers  and  their  trainees.  All  too  often  this  results  in  tools  that  represent  the  state  of  the 
art  in  technology  but  do  not  meet  the  needs  of  the  warfighter. 

To  overcome  these  obstacles,  developers  of  training  technology  need  to  adopt  a  comprehensive, 
user-centered  approach  to  training  in  sociocultural  understanding.  Practitioners  (cultural  trainers) 
and  technologists  (researchers  and  engineers)  must  work  together  to  achieve  a  common  goal, 
rather  than  to  advance  their  preferred  approaches  to  addressing  the  problem.  This  often  requires  a 
third  party  "translator"  who  can  not  only  understand  and  elicit  end  user  requirements  and 
preferences,  but  also  translate  these  into  usable  guidance  for  the  developers  of  the  technology. 
This  'training  +  technology'  marriage  will  both  improve  the  quality  of  the  training  program  and 
promote  the  acceptance  and  use  of  technological  tools  in  training  centers— especially  those  that 
participated  in  and  shaped  the  design  and  development  of  the  technological  solutions.  For 
example,  361  Interactive's  work  with  the  JFKSWCS  leadership  and  trainers  to  tailor  CultureGear  and 
integrate  it  into  their  training  curriculum  places  the  end  users  at  the  forefront  of  the  development 
process,  validating  their  critical  importance,  and  ensuring  that  their  needs  will  be  addressed 
upfront.  Ultimately,  only  a  deep  and  thorough  understanding  of  the  needs,  desires,  and  challenges 
of  the  operational  community  can  ensure  that  the  military  will  accept,  and  indeed  embrace, 
training  technologies. 
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Section  Two:  Detecting 

Sensing  sociocultural  factors  and  elements  in  an 
environment 
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Capability  Area  2:  Detecting 

"Capabilities  to  discover,  distinguish,  and  locate  operationally  relevant  sociocultural  signatures 
through  the  collection,  processing,  and  analysis  of  sociocultural  behavior  data"  (Schmorrow,  2011, 
P-  42). 

Sociocultural  signatures  that  potentially  have  operational  relevance  cover  a  vast  domain, 
encompassing  the  perceptions,  sentiments,  attitudes,  and  behaviors  of  various  populations  of 
interest.  Challenges  in  developing  a  robust  capability  to  detect  these  signatures  are  equally 
daunting.  Such  a  capability  depends  on  processing  and  then  analyzing  voluminous  collections  of 
relevant  data  sources,  with  data  types  ranging  from  raw  sensor  measurements  to  text,  image,  and 
audio  data,  to  qualitative  assessments  about  attitudes.  The  data  cover  varying  time  spans,  and 
describe  phenomena  at  a  variety  of  scales  and  levels  of  detail.  Further,  detection  requires  analysts 
to  resolve  many  technical  issues,  including  the  need  to  isolate  meaningful  signals  in  a  deluge  of 
noise,  manage  both  structured  and  unstructured  data,  process  data  streams  in  real  time,  and 
identify  rare  events.  The  chapters  in  this  section  describe  state-of-the-art  capabilities  available  to 
meet  some  of  these  challenges,  as  well  as  emerging  capabilities  that  are  likely  to  become  useful  in 
the  near  future. 


Data  Processing 

Irvine's  chapter,  "Transforming  data  into  information:  enabling  detection  and  discovery  for 
sociocultural  analysis,"  reviews  the  issues  involved  in  transforming  raw  data  into  the  information 
that  researchers  need  to  detect  interesting  sociocultural  patterns.  The  discussion  focuses  primarily 
on  four  types  of  data  sources:  surveys,  social  media,  imagery,  and  video.  Surveys  provide  controlled 
methods  for  collecting  data  about  individuals  and  societies.  Social  media  supply  direct  and  timely 
data  about  the  opinions  of  individuals,  as  well  as  the  unfolding  of  events.  Imagery  is  becoming  an 
increasingly  valuable  source  of  data  about  societies  in  general  as  well  as  local  attitudes  and 
behaviors.  Video  contains  data  that  can  be  used  to  analyze  temporal  events  and  detect  both  simple 
activities  and  complex  behaviors. 

Each  type  of  data  presents  its  own  set  of  challenges,  and  the  data  processing  methods  available  to 
address  those  challenges  have  varying  levels  of  maturity.  Numerous  methods  are  available  for 
analyzing  the  data  and  drawing  inferences  from  the  results.  This  chapter  pays  special  attention  to 
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two  somewhat  recent  paradigms  for  conducting  analysis:  social  network  analysis,  which  represents 
the  relationships  among  individuals  and  groups  and  facilitates  analysis  of  the  interactions  among 
sociocultural  actors;  and  change  detection  methods  that  can  identify  significant  shifts  in  a  pattern 
of  activity  or  behavior. 


Computational  Modeling 

In  "Current  trends  in  the  detection  of  sociocultural  signatures:  data-driven  models,"  Sanfilippo, 
Bell,  and  Corley  address  detection  of  sociocultural  activities  and  events  from  a  modeling 
perspective.  The  prominent  theme  here  is  that  analysts  must  "harvest"  data  in  forms  that  enable 
them  to  build,  calibrate,  and  run  computational  models  of  sociocultural  phenomena.  This  means 
that  parameters  of  data  records  relevant  to  the  content  categories  of  interest  must  first  be 
extracted  and  measured  to  produce  sociocultural  data  signatures.  Many  techniques  are  available  to 
assist  in  this  process.  Computational  models  then  use  the  data  signatures  to  characterize  the 
behavioral  patterns  of  interest  in  the  underlying  data.  While  there  are  many  different  kinds  of 
computational  models,  all  can  be  viewed  in  terms  of  sociocultural  model  signatures  that  specify 
how  to  detect  and  assess  sociocultural  patterns. 

Social  media  content  provides  some  unique  challenges  and  opportunities  for  computational 
modeling.  The  authors  review  new  data  harvesting  methods  specifically  designed  to  address  those 
challenges,  along  with  examples  of  the  kinds  of  models  that  can  use  social  media  data  to  detect 
sociocultural  patterns.  They  also  provide  a  brief  discussion  about  how  detection  models  can  be 
used  as  a  starting  point  to  help  make  predictions. 


Visualization 

Even  after  the  relevant  sociocultural  data  have  been  processed,  and  analysts  have  drawn 
inferences  with  the  aid  of  computational  models  or  other  analytical  tools,  the  sociocultural 
phenomena  we  wish  to  discern  from  the  data  might  still  not  be  self-evident.  Detection  typically 
requires  some  further  examination  and  interpretation  of  the  analytic  results.  As  Fricker,  Buttrey, 
and  Evans  point  out  in  their  chapter,  "Visualization  for  sociocultural  signature  detection," 
visualization  techniques  provide  invaluable  support  for  this  final  step  in  discovering  the 
sociocultural  signatures  associated  with  the  data  and  identifying  when  those  signatures  change. 

This  chapter  provides  a  broad  overview  of  the  most  widely  used  approaches  to  visualizing  data.  The 
authors  devote  particular  attention  to  the  methods  best  suited  for  the  data  types  relevant  to  the 
detection  of  sociocultural  signatures:  networks,  geographic  information,  survey  data,  linguistic 
data,  and  social  media  data.  While  the  methods  discussed  are  applicable  to  the  other  operational 
capability  areas  (understanding,  forecasting,  and  mitigation)  as  well,  this  chapter  emphasizes  how 
those  capabilities  are  employed  for  detection.  Accordingly,  the  discussion  highlights  exploratory 
data  analysis,  as  well  as  simplifications  that  help  focus  attention  on  differences  across  a  data 
collection  or  on  changes  in  the  data  that  occur  over  time. 


Section  introduction:  Detecting  |  109 


Training 

Sociocultural  signatures  must  be  interpreted  in  context.  Consequently,  detecting  sociocultural 
signatures  requires  more  than  discovering  indicators  of  sociocultural  phenomena  in  data  or  in 
analytic  results.  It  also  demands  a  deep  understanding  of  the  real-world  signatures  associated  with 
the  findings  in  the  data,  why  they  are  important,  and  what  they  mean.  In  "Cross-cultural  training 
and  education  for  detection,"  Glazer,  Saner,  Barnes,  and  Pavisic  explain  how  sociocultural  analysis 
can  aid  understanding  of  how  and  why  the  observed  phenomena  manifest  themselves  in  a 
particular  form  in  a  given  setting.  They  also  point  out  the  importance  of  cross-cultural  training  and 
education  as  prerequisites  for  conducting  robust  sociocultural  analysis. 

Knowledge  about  culture  provides  the  context  needed  to  achieve  the  level  of  sociocultural 
awareness  required  for  detection.  This  chapter  provides  an  overview  of  culture  and  culture-general 
concepts,  reviews  findings  from  state-of-the-art  research  on  culture,  and  identifies  gaps  in  current 
analytic  practice.  Glazer  and  colleagues  also  offer  observations  about  implications  regarding  the 
most  effective  methods  and  designs  to  train  and  educate  intelligence  professionals  to  increase 
their  sociocultural  awareness. 

As  these  brief  descriptions  show,  analysts  can  apply  a  wide  variety  of  methods  and  technologies  to 
help  them  discover,  distinguish,  and  locate  sociocultural  signatures.  More  work  remains,  however, 
to  make  these  capabilities  routinely  available  for  operational  use,  and  each  chapter  indicates  some 
promising  directions. 
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1.  Introduction 

Nowadays  it  seems  impossible  to  open  any  recent  issue  of  a  technical  journal,  trade  newsletter,  or 
popular  science  publication  that  does  not  contain  at  least  one  article  about  "big  data."  To  be  sure, 
digital  data  is  accumulating  at  a  staggering  rate  -  roughly  80  petabytes  an  hour,  more  than  10 
times  larger  than  the  holdings  of  the  Library  of  Congress  (Smolan  &  Erwitt,  2012).  The  combination 
of  systematic  data  collection  by  commercial  and  government  organizations,  the  emergence  of 
unstructured  digital  data  sources  from  news  and  social  media,  and  advances  in  sensor  technology 
has  led  to  this  explosion  of  data.  For  researchers  seeking  to  detect  interesting  sociocultural 
patterns,  the  volume  of  data  presents  both  opportunities  and  challenges.  The  opportunities  stem 
from  the  availability  of  data  to  describe  many  aspects  of  the  daily  life  of  individuals  and  societies. 
The  challenge  is  distilling  this  massive  quantity  of  bits  and  bytes  into  meaningful  information  that 
supports  deeper  analysis. 

In  approaching  the  derivation  of  understanding  from  raw  data,  we  present  a  conceptual  framework 
that  considers  the  basic  processing  components  (Figure  1).  The  raw  data  can  originate  from  any 
number  of  sources:  surveys,  social  media,  news  reports,  and  sensor  data.  One  can  exploit  each  of 
the  sources  individually,  but  greater  benefits  often  arise  from  synergistic  analysis  of  multiple 
sources,  i.e.,  information  fusion  (Hall,  1992;  Klein,  1999).  The  timeline  for  developing  inferences 
from  the  data  depends  on  the  specific  mission  or  application.  In  some  tactical  settings  users  may 
need  to  derive  inferences,  including  warnings  or  alerts,  under  demanding  time  constraints.  Deeper 
analysis,  including  forensics,  might  require  days,  weeks,  or  even  months.  The  ways  in  which  users 
might  interact  with  the  analytic  tools  differ  for  these  two  types  of  missions. 

1.1.  Data  Sources 

Many  sources  of  data  provide  useful  information  about  sociocultural  issues,  either  directly  or 
indirectly.  Traditional  tools  for  social  science  research  have  included  direct  observations  and 
surveys,  as  well  as  published  media  such  as  news  reports.  More  recently,  social  media  have 
changed  the  nature  of  the  data,  because  now  the  actors  in  a  society  provide  information  directly 
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rather  than  through  the  filter  of  an  observer.  Lastly,  researchers  are  starting  to  explore  sensor  data, 
such  as  imagery  that  measures  physical  phenomena,  as  a  source  of  information. 

1.2.  Sources  to  Evidence 

Raw  data  generally  have  only  limited  value.  Some  processing  is  necessary  to  extract  useful 
information  from  the  original  sources.  The  methods  for  transforming  sources  of  raw  data  into 
information  or  evidence  depend  on  both  the  nature  of  the  source  data  and  the  information 
required  by  the  end  user.  For  example,  processing  the  results  of  a  survey  to  assess  support  for  a 
political  candidate  can  be  simple.  The  tabulation  of  the  responses  gives  an  initial  indication  of 
findings,  and  more  sophisticated  processing  would  weight  the  responses  in  accordance  with  the 
sample  design.  Still  more  sophisticated  analysis  might  attempt  to  assess  the  veracity  of  responses 
or  impute  values  for  missing  responses  based  on  answers  to  related  questions  (Cooley  &  Lohnes, 
1971).  More  challenging  data  sources  include  unstructured  and  semi-structured  text.  For  example, 
a  researcher  might  want  to  explore  the  level  of  support  in  neighboring  Arab  states  for  the  Syrian 
rebels  seeking  to  overthrow  President  Assad  by  examining  posts  on  social  media  sites.  Turning  the 
text  data  into  relevant  information  involves  performing  sentiment  analysis  on  relatively 
unstructured  text  that  could  include  slang  and  country-specific  references.  This  type  of  analysis 
presents  far  more  challenges  and  the  tools  available  today,  while  promising,  are  less  than  perfect. 

Transforming  data  into  information  can  also  involve  a  data  association  problem.  If  multiple  sources 
of  data  are  available,  which  pieces  go  together?  For  example,  one  might  find  a  photograph  of  a 
demonstration  on  Flickr  and  tweets  about  demonstrators  being  arrested.  Are  these  two  chunks  of 
data  related?  Associating  two  data  elements  often  relies  on  some  co-occurrence  in  space  and  time. 
Reports  that  refer  to  the  same  location  and  date  are  likely  to  describe  the  same  event.  When  such 
information  is  limited,  estimation  of  these  attributes  is  necessary  and  the  data  association  carries  a 
level  of  ambiguity.  Other  methods  of  data  association  might  rely  on  named  entities;  for  example, 
texts  that  refer  to  the  same  person  or  event  might  belong  together. 

1.3.  Evidence  to  Inference 

Armed  with  useful  information,  researchers  must  next  draw  inferences  based  on  the  data.  Various 
inferences  are  possible,  depending  on  the  research  questions  and  the  nature  of  the  data.  Common 
types  of  inference  include: 

Discovery:  Uncovering  a  new  phenomenon  or  relationship.  The  researcher  has  access  to 
information  that  has  not  been  fully  exploited  and  should  have  some  working  hypotheses,  but  the 
exploration  of  the  data  could  reinforce  or  refute  these  hypotheses.  More  important,  the 
exploration  can  suggest  new  relationships  to  consider.  Data  mining  offers  a  set  of  tools  commonly 
used  for  discovery. 

Estimation  and  modeling:  Determining  the  quantity,  strength,  or  extent  of  a  behavior,  attribute,  or 
relationship.  One  may  want  to  estimate  the  size  of  a  particular  ethnic  group,  for  example,  or  the 
relationship  between  religious  affiliation  and  political  party.  Survey  research  provides  a  traditional 
tool  for  this  type  of  analysis.  The  formulation  and  testing  of  models  often  accompany  estimation. 
Consider,  for  example,  the  likelihood  of  a  particular  district  voting  for  particular  party  in  the  next 
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election.  A  simple  model  might  explore  the  relationship  between  party  affiliation  and  other  factors, 
such  as  ethnicity,  religion,  and  income. 


Change  detection:  Identifying  a  departure  from  past  norms  or  behaviors.  The  ability  to  detect 
genuine  changes  in  a  society  and  to  distinguish  real  changes  from  random  fluctuations,  is  critical  to 
identifying  emerging  patterns.  Early  detection  of  the  Arab  Spring,  for  example,  could  have 
benefited  numerous  decision  makers  by  signaling  the  possibility  of  regime  changes  and  the 
implications  for  trade,  commerce,  and  diplomacy.  However,  many  societies  have  experienced 
various  levels  of  unrest  without  experiencing  the  changes  of  government  that  occurred  in  Tunisia 
or  Egypt.  What  tools  help  us  distinguish  true  changes  from  small  perturbations  in  a  society? 

1.4.  User  Understanding 

Researchers  could  perform  the  preceding  steps  in  processing  via  automated  methods,  user 
manipulation  of  the  data,  or  a  combination  of  human  and  automated  processing.  The  final  step 
requires  a  human  to  interpret  the  results  of  the  processing.  Deeper  analysis  and  understanding  can 
only  come  when  the  user  examines  the  inferences  produced  by  processing,  displaying,  and 
visualizing  the  information,  and  explores  the  relationships.  A  separate  chapter  in  the  Detecting 
section  of  this  book  explores  data  visualization  in  more  depth.  For  the  purposes  of  the  present 
discussion,  however,  it  is  important  to  realize  that  users  need  tools  and  methods  for  querying  the 
data,  displaying  information,  and  understanding  the  data  provenance. 
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Figure  1.  Processing  steps  for  transforming  data  into  actionable  information. 


Section  2  of  this  chapter  focuses  on  the  first  portion  of  Figure  1,  exploring  several  types  of  data 
relevant  to  sociocultural  analysis  and  some  of  the  methods  for  processing  these  data.  Section  3 
describes  the  development  of  inferences  from  the  processed  data,  as  portrayed  in  the  middle  box 
in  the  figure.  Many  methods  support  data  analysis,  fusion,  and  inference,  but  we  concentrate  on 
selected  topics  relevant  to  sociocultural  modeling.  The  chapter  concludes  with  some  discussion  of 
the  technology  gaps. 
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2.  Data  Sources  and  Information 

Almost  any  source  of  data  contains  information,  either  direct  or  indirect,  about  the  society  and 
culture  that  produced  the  data.  Certain  data  sources,  however,  have  emerged  as  particularly  useful 
for  understanding  sociocultural  phenomena.  In  this  chapter,  we  focus  on  four  specific  sources: 
survey  data,  social  media,  imagery  data,  and  video  data.  Some  of  the  data  sources  we  discuss  are 
also  addressed  in  other  chapters  in  this  volume;  in  the  present  chapter  we  focus  on  the  use  of  data 
and  information  for  purposes  directly  related  to  detection. 

2.1.  Survey  Data:  Controlled  Methods  for  Data  Collection 

A  basic  approach  to  understanding  individuals  and  societies  consists  of  asking  them  questions 
directly.  A  conversation  with  one  individual  is  termed  an  interview.  While  interviews  can  be 
revealing,  the  information  is  specific  to  the  person  being  interviewed.  A  broader  discussion  with 
multiple  participants,  in  the  form  of  a  focus  group,  offers  a  systematic  way  to  elicit  a  broader  set  of 
information.  Extending  the  concept  to  a  structured  approach  that  elicits  information  from  many 
respondents  moves  the  process  into  the  realm  of  survey  research.  Depending  on  the  research 
questions,  a  trade-off  exists  between  the  rich  narrative  available  from  open-ended  questions  with 
free-text  responses  and  specific,  focused  questions  amenable  to  quantitative  analysis. 

Surveys  have  many  advantages.  They  offer  a  direct  measurement  of  attitudes  and  opinions,  the 
methodology  has  a  high  degree  of  transparency  and  repeatability,  the  data  acquired  are  amenable 
to  analysis  using  standard  statistical  tools,  and  the  results  and  findings  can  be  represented 
quantitatively.  Some  of  the  drawbacks  arise  from  the  costs  of  conducting  surveys,  the  challenges  of 
designing  good  survey  instruments,  and  the  difficulty  of  performing  surveys  in  some  parts  of  the 
world.  Not  all  respondents  may  feel  comfortable  giving  honest  responses  to  certain  types  of 
questions,  either  because  of  cultural  norms  or  because  of  safety  concerns.  In  some  cases, 
researchers  can  devise  questionnaires  that  address  sensitive  issues  in  ways  that  make  the 
respondent  more  comfortable;  in  other  cases,  it  may  be  impossible  to  obtain  responses  to  certain 
questions.  For  example,  in  a  country  with  a  regime  known  for  suppressing  dissent,  survey 
respondents  are  unlikely  to  give  honest  answers  to  questions  about  support  for  the  government. 

Numerous  books  and  tutorials  discuss  methods  for  conducting  surveys,  and  the  use  of  survey 
methods  has  grown  in  recent  years.  Cross-cultural  survey  research  requires  particular  care  due  to 
various  methodological  pitfalls  (University  of  Michigan,  2011).  A  number  of  commercial 
organizations  can  provide  the  necessary  services  for  anyone  wishing  to  conduct  a  survey. 
Nevertheless,  researchers  must  understand  the  process  of  designing  and  running  the  survey. 

Research  objectives:  Identifying  the  research  objectives  is  critical.  What  does  the  researcher  want 
to  learn  about  the  society  and  its  members?  Is  the  research  topic  even  amenable  to  survey 
methods? 

Development  of  survey  instruments:  Designing  the  questionnaire  is  generally  an  iterative  process. 
Questions  mean  different  things  to  different  people.  Through  testing  and  interviews,  the 
researcher  must  refine  the  questions  to  ensure  that  responses  will  be  informative.  The  willingness 
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of  respondents  is  another  important  consideration.  How  much  time  does  the  survey  take?  Are 
respondents  compensated  for  their  time? 

Translation  and  pilot  testing:  What  language(s)  do  the  respondents  speak?  In  some  countries,  it 
may  be  necessary  to  conduct  the  survey  in  multiple  languages.  Are  the  questions  asked  of  the 
different  language  groups  comparable?  Pilot  testing  of  the  survey  instruments  can  help  address 
this  concern,  but  a  fundamental  understanding  of  the  language  issues  is  vital. 

Sample  design:  Generally,  surveys  are  conducted  to  make  estimates  about  a  particular  society. 
Does  the  sample  design  for  the  survey  reflect  these  objectives?  For  example,  if  the  research  covers 
the  country  as  a  whole,  does  the  survey  sample  all  geographic  regions?  Does  it  include  a  mix  of 
urban  and  rural  respondents?  An  extensive  literature  exists  on  sample  design  and  weighted 
sampling  units  (Cochran,  1977;  Thompson  &  Seber,  1996).  In  particular,  methods  exist  for 
partitioning  the  population  into  strata  and  using  different  sampling  rates  for  each  stratum. 
Depending  on  the  objective  of  the  survey,  these  stratified  sampling  methods  can  be  cost-effective 
methods  for  conducting  the  research. 

Data  collection:  The  actual  field  work  of  conducting  the  survey  is  labor  intensive.  Depending  on  the 
region  of  the  world,  safety  of  the  interviewers,  the  costs  of  the  operation,  and  the  difficulty  in 
obtaining  complete  responses  pose  varying  types  of  challenges. 

Data  cleaning:  Respondents  often  do  not  answer  all  of  the  questions  posed  by  the  interviewer.  In 
addition,  the  type  of  question  can  introduce  ambiguity  in  the  responses,  especially  if  open 
responses  (rather  than  pre-coded  answers)  are  allowed.  The  data  cleaning  step  verifies  the  quality 
and  correctness  of  the  information  and  assigns  appropriate  codes  for  missing  responses. 

Unstructured  data:  Open-ended  survey  questions  and  focus  group  discussions  can  generate  a  large 
amount  of  useful  information,  but  the  data  are  generally  captured  as  unstructured  text.  These  data 
can  be  analyzed  through  the  labor-intensive  process  of  reviewing  all  of  the  free  text  to  identify 
useful  information.  Alternatively,  researchers  can  employ  automated  text  analytics  to  extract 
content,  identify  entities,  or  analyze  sentiment.  While  these  automated  methods  can  prove  cost 
effective  for  processing  large  volumes  of  data,  they  contain  some  inherent  level  of  error. 

Data  analysis:  Well-developed  statistical  methods  exist  for  analyzing  data.  The  responses  to  survey 
question  are  frequently  captured  as  Yes/No  or  multiple  choice  responses.  Researchers  can  use 
methods  for  categorical  data  analysis,  including  log-linear  modeling,  to  analyze  individual  questions 
and  relationships  across  questions  (Agresti,  2012). 

Generalizing  from  the  survey:  The  goal  of  survey  research  is  to  develop  an  understanding  that 
extends  to  a  larger  population.  The  validity  of  this  generalization  depends  on  the  sampling  method 
used  to  obtain  survey  responses.  If  the  research  uses  stratified  sample  methods,  estimated  rates, 
confidence  intervals,  and  tests  of  hypotheses  must  be  handled  appropriately  (Fay,  1990;  Wolter, 
1985). 
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Well-designed  surveys  can  provide  tremendous  insights  into  a  society.  One  example  is  an  extensive 
set  of  public  opinion  data  collected  in  sub-Saharan  Africa  over  several  years  under  the 
Afrobarometer  Program.  The  Afrobarometer  is  a  collaborative  enterprise  of  the  Centre  for 
Democratic  Development  in  Ghana,  the  Institute  for  Democracy  in  South  Africa,  and  the  Institute 
for  Empirical  Research  in  Political  Economy  with  support  from  Michigan  State  University  and  the 
University  of  Cape  Town,  Center  of  Social  Science  Research.  Each  Afrobarometer  survey  collects 
data  about  individual  attitudes  and  behavior,  including  indicators  relevant  to  developing  societies. 
These  issues  are  summarized  in  Table  1. 


Table  1.  Issues  addressed  in  Afrobarometer  surveys 


Topic  Area 

Description 

Democracy 

Popular  understanding  of,  support  for,  and  satisfaction  with 
democracy,  as  well  as  any  desire  to  return  to,  or  experiment  with, 

authoritarian  alternatives. 

Governance 

The  demand  for,  and  satisfaction  with,  effective,  accountable  and  clean 
government;  judgments  of  overall  governance  performance  and  social 
service  delivery. 

Livelihoods 

How  do  families  survive?  What  variety  of  formal  and  informal  means  do 
they  use  to  gain  access  to  food,  shelter,  water,  health,  employment  and 
money? 

Macroeconomics 

and  Markets 

Citizen  understandings  of  market  principles  and  market  reforms  and 
their  assessments  of  economic  conditions  and  government 
performance  at  economic  management. 

Social  Capital 

Whom  do  people  trust?  To  what  extent  do  they  rely  on  informal 

networks  and  associations?  What  are  their  evaluations  of  the 

trustworthiness  of  various  institutions? 

Conflict  and  Crime 

How  safe  do  people  feel?  What  has  been  their  experience  with  crime 

and  violence? 

Participation 

The  extent  to  which  ordinary  people  join  in  development  efforts, 
comply  with  the  laws  of  the  land,  vote  in  elections,  contact  elected 
representatives,  and  engage  in  protest.  The  quality  of  electoral 
representation. 

National  Identity 

How  do  people  see  themselves  in  relation  to  ethnic  and  class 
identities?  Does  a  shared  sense  of  national  identity  exist? 
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The  survey  data  provide  a  rich  portrait  of  societal  attitudes  across  several  countries  and  multiple 
regions  within  each  country.  For  some  countries,  surveys  have  been  repeated  over  multiple  years, 
giving  a  temporal  characterization  of  shifting  attitudes  and  opinions.  For  instance,  one  survey 
question  explored  attitudes  concerning  the  importance  of  having  a  democratic  society  and  the 
freedom  to  criticize  the  government.  Responses  show  distinct  patterns  by  country  and,  to  a  lesser 
extent,  within  countries  (Figure  2).  Deeper  analysis  reveals  that  within  a  country,  attitudes  vary 
with  the  level  of  urbanization  and  the  economic  status  of  respondents. 


Zimbabwe  Botswana 


Figure  2.  Democratic  society  and  freedom  to  criticize  the  government.  Responses  by  country  and 
region  from  Afrobarometer  data  (Afrobarometer,  2013). 


2.2.  Social  Media:  Text  and  Multimedia  Data 

The  use  of  social  media  has  exploded  in  recent  years,  making  multiple  sources  of  data  available  to 
anyone  who  can  harvest  all  the  bits  and  bytes.  Analysis  of  social  media  data  appeals  for  several 
reasons:  it  provides  direct  access  to  people's  comments  and  opinions,  the  information  is  readily 
available,  and  very  little  time  elapses  between  the  unfolding  of  an  event  and  the  corresponding 
reactions  in  social  media.  Because  social  media  offer  direct  links  to  events  as  they  happen, 
numerous  researchers  are  exploring  ways  to  leverage  this  data  for  understanding  a  wide  range  of 
geopolitical,  economic,  and  sociocultural  issues. 

The  data  sources  include  exchanges  of  short  text  messages  (such  as  Twitter);  longer  unstructured 
text  messages  that  appear  on  Facebook,  blogs,  and  other  sites;  and  imagery  data  uploaded  to  sites 
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such  as  Flickr  and  YouTube.  Many  of  the  popular  sites  are  specific  to  a  single  country  or  limited 
geographical  region.  For  example,  people  in  China  have  no  access  to  Facebook,  Twitter,  and 
YouTube.  Instead,  Weibo,  Renren,  Youku,  and  Kaixin  fill  the  niche.  In  addition,  the  information 
posted  to  social  media  sites  can  be  in  a  variety  of  languages,  including  slang  or  common  shorthand 
terms  that  members  of  the  online  community  know  and  use.  Methods  for  information  extraction 
must  be  designed  and  tuned  accordingly  (Bollen,  Mao,  &  Pepe,  2011). 

Automated  methods  for  extracting  useful  information  from  these  sources  must  address  a  number 
of  issues:  access  to  the  data,  data  volume,  language,  and  the  unstructured  nature  of  the  data.  A 
growing  body  of  research  explores  methods  of  analysis  for  unstructured  and  semi-structured  text. 
Some  of  the  technical  challenges  include  automated  methods  for  entity  and  topic  identification, 
content  extraction,  sentiment  analysis,  and  trend  analysis  (Feldman  &  Sanger,  2006;  Manning  & 
Schuetze,  1999). 

One  interesting  area  of  investigation  combines  the  content  analysis  of  social  media  with  geospatial 
data.  Along  with  the  actual  content  of  tweets,  blog  posts,  and  images  on  Flickr,  the  metadata  often 
include  location  information.  Combining  this  location  information  with  simple  content  analysis  can 
reveal  patterns  of  spatial  distribution  and  movement.  Since  many  people  use  mobile  devices,  such 
as  smart  phones,  even  rapidly  changing  events  have  become  amenable  to  geospatial  analysis 
through  social  media.  An  interesting  example  (Stefanidis,  2013)  depicts  the  spatial  analysis  of 
Twitter  data  related  to  the  rebel  activity  in  Syria  (Figure  3).  As  expected,  the  heat  map  shows 
intense  activity  in  Syria  and  high  levels  in  surrounding  countries  in  the  Middle  East.  Looking 
globally,  however,  we  see  pockets  of  activity  in  North  America  and  Europe,  as  well  as  smaller  hot 
spots  in  Argentina  and  Australia.  This  suggests  active  Syrian  communities  or  supporters  in  specific 
parts  of  the  world  outside  of  Syria.  Researchers  can  now  perform  deeper  analysis  of  the  content 
and  related  data  sources  to  understand  the  nature  of  these  communities. 


Transforming  data  into  information  |  119 


!<*»» 


Map  Satellite 


North 

Atlantic 

Ocean 


Finland 

Sweden 

Norway 

Poland 

Germany  Ukraine 


Afghanistan 

'“>9  Iran 

Algeria  Ljbya  Egypt  Pakistan 

^  _ — i.  ^  India 


Mali  Niger  Sudan 

Chad 


Kenya 

Indonesia  PaDua 

mzania  £uir 


Indian 

Ocean 


mn k 


Map  data  ©201 3  MapLink  -  Terms  of  Use 


Figure  3.  Social  media  map  of  the  Syrian  community.  Adapted  with  permission  from  Stefanidis 
(2013). 


Exploiting  social  media  data  represents  a  fundamental  change  in  data  acquisition  and  analysis. 
Other  methods  for  data  collection,  such  as  surveys  or  sensor-based  methods,  rely  on  designed  data 
collection  plans  for  obtaining  the  information.  Collection  of  social  media  is  entirely  opportunistic: 
the  target  populations  generate  the  data  through  their  normal  activities  and  the  researcher  can 
harvest  this  information  for  subsequent  analysis.  With  traditional  methods,  the  target  population 
and  the  data  acquisition  process  are  distinct.  With  social  media,  the  objects  of  study  are  also  the 
source  of  the  data.  In  effect,  members  of  the  target  population  also  function  as  the  "sensor." 

Social  media  offer  a  number  of  advantages  over  traditional  data  sources.  Because  the  data 
acquisition  process  is  built  into  the  society,  no  separate  observation  process,  such  as  conducting  a 
survey  or  flying  a  sensor,  is  necessary.  Thus,  acquiring  social  media  data  can  be  much  less 
expensive  than  other  methods.  Furthermore,  social  media  offer  a  direct  method  of  observation; 
attitudes  and  opinions  can  be  captured  without  any  of  the  filtering  or  interpretation  inherent  in 
other  methods.  Lastly,  the  data  source  can  be  extremely  timely.  Participants  in  an  event  often 
tweet  immediately,  enabling  researchers  to  observe  events  in  near-real  time. 

Despite  these  advantages,  analyzing  social  media  data  involves  some  serious  challenges.  The  first 
stems  from  the  nature  of  the  data.  Observations  are  largely  unstructured  and  each  small  packet  of 
data,  such  as  a  tweet  or  a  blog  post,  contains  very  little  information.  The  power  of  this  data  source 
rests  in  the  accumulation  of  evidence  across  a  large  set  of  observations.  Therein  lies  the  challenge. 
The  research  must  harvest  the  data  and  process  it  in  a  way  that  sifts  out  the  meaningful  content 
from  the  morass  of  data.  Some  social  media  data  include  metadata,  such  as  time  and  location  tags, 
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that  can  assist  in  this  process.  However,  researchers  must  extract  content,  entities,  and  sentiment 
from  unstructured  text  to  realize  most  of  the  benefits  of  these  sources. 

Another  limitation  of  social  media  data  is  the  inherent  sampling  bias  in  this  source.  The  people  who 
use  social  media  are  generally  younger,  better  educated,  and  technologically  savvy.  They  have 
access  to  computers,  smart  phones,  and  iPads  and  are  more  likely  to  live  in  or  close  to  urban  areas. 
Consequently,  observations  drawn  from  this  source  probably  under-represent  older  generations, 
residents  of  rural  areas,  or  the  poor.  Despite  these  limitations,  however,  social  media  offer  an 
exciting  new  window  into  the  attitudes  and  workings  of  a  society. 

2.3.  Sensor  Data:  Imagery 

The  application  of  remote  sensing  and  geographic  information  systems  (GIS)  to  the  social  sciences 
is  an  emerging  research  area  (Blumberg  &  Jacobson,  1997;  Crews  &  Walsh,  2009;  Goodchild, 
Anselin,  Appelbaum,  &  Harthorn,  2000;  Taubenbock  et  al.,  2009).  "Remote  sensing  can  provide 
measures  for  a  number  of  variables  associated  with  human  activity— particularly  regarding  the 
environmental  consequences  of  various  social,  economic,  and  demographic  processes"  (National 
Research  Council,  1998,  p.  116).  Recognizing  that  people's  behavior  and  values  shape,  and  are 
shaped  by,  the  environment  in  which  they  live,  researchers  have  explored  a  number  of  issues, 
including  sociocultural  and  economic  attributes  of  the  population,  ethnography,  and  land  use  (Fox, 
2003;  Jiang,  2003).  A  review  of  the  literature  indicates  that  investigators  are  exploring  the 
connection  between  remote  sensing  and  social  science  issues.  Established  research  focuses  on 
urban  studies,  demography,  archaeology,  land  use  and  land  cover,  and  war  and  conflict  studies 
(Hall,  2010).  Recent  and  emerging  areas  of  investigation  include  deeper  exploration  of  economic 
and  governmental  attributes. 

Several  recent  studies  have  demonstrated  that  imagery  collected  from  space-based  or  airborne 
imaging  assets  can  reveal  useful  information  about  societies  and  the  environment  in  which  they 
live.  The  LandScan  Project  at  Oak  Ridge  National  Laboratory  has  developed  methods  for  estimating 
populations  and  population  distribution  on  a  global  basis  (Landscan,  2013;  Bhaduri,  Bright, 
Coleman,  &  Urban,  2007;  Cheriyadat,  Bright,  Bhaduri,  &  Potere,  2007;  Vijayaraj,  Bright,  &  Bhaduri, 
2007;  Vijayaraj,  Cheriyadat,  Sallee,  Colder,  Vatsavai,  Bright,  &  Bhaduri,  2008).  This  program  has 
demonstrated  the  practical  application  of  remote  sensing  methods  for  mapping  human 
settlements  and  analyzing  population  movements  overtime.  A  recent  study  of  the  urban  landscape 
in  Guatemala  (Owen,  2012)  represents  a  more  focused  application  of  remote  sensing.  This  work 
made  an  important  contribution  through  an  assessment  that  distinguished  informal  (slum)  and 
formal  (planned)  settlements  using  high-resolution  imagery.  Finally,  Min  made  a  novel  use  of 
"nightlights"3  data  to  assess  political  and  military  activity  remotely  (Min,  2008a). 


3  Defense  Meteorological  Satellite  Program's  Operational  Linescan  System  (DMSP-OLS)  captures  night  imagery 
of  the  earth,  which  can  reveal  lighting  patterns  arising  from  human  settlements. 
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Table  2.  Research  issues  and  potential  observables  to  explore 


Research  Issue 

Potential  Observables 

Income  and  Economic 

Development: 

House  sizes:  average  size,  range  of  sizes 

Presence  of  motor  vehicles 

Physical  infrastructure 

Agriculture:  extent  and  mix  of  cultivation,  crop  health,  presence 

and  extent  of  livestock 

Centrality  and  Decision 
Authority 

Road  network;  lines  of  communication 

Physical  infrastructure  (bridges,  paved  roads,  schools,  mosques) 

Social  Capital 

Community  infrastructure  and  prevalence  of  meeting  places  and 
institutions  (schools,  places  of  worship). 

Communications  infrastructure  (roads) 

Social  science  research  suggests  several  ways  in  which  local  attitudes  and  behaviors  may 
correspond  to  phenomena  observable  in  overhead  imagery.  Although  this  set  is  not  exhaustive, 
certain  issues  have  a  foundation  in  social  science  research  and  a  logical  connection  to  observable 
phenomena  (Table  2).  The  theory  suggests  natural  hypotheses  to  examine  in  three  areas:  income 
and  economic  development,  centrality  and  decision  authority,  and  social  capital.  First,  overhead 
imagery  reveals  numerous  indicators  of  economic  status  (housing,  vehicles,  crop  land,  livestock, 
and  infrastructure).  Studies  have  explored  the  relationship  between  remote  sensing  data  and  the 
economy  (Elvidge,  Baugh,  Kihn,  Kroehl,  Davis,  &  Davis,  1997;  Irvine  et  al.,  2013).  Second,  both 
higher  income  and  equitable  distribution  of  income  are  associated  with  good  governance. 
Observables  associated  with  economic  well-being,  including  measures  of  wealth  distribution,  also 
serve  as  relevant  indicators  of  governance.  The  transportation  and  communication  infrastructure 
provides  indicators  of  expected  levels  of  social  interactions.  Third,  high  social  capital  has  been 
linked  to  good  governance  (Bardhan,  Bowles,  &  Gintis,  2000;  Bowles  &  Gintis,  2002),  but  observing 
meaningful  measures  that  correlate  with  social  capital  poses  a  challenge.  Evidence  of  economic 
growth  can  be  associated  with  higher  levels  of  social  capital  (Knack  &  Keefer,  1997).  Although  social 
cohesion  and  connectedness  cannot  be  measured  directly,  durable  institutions  (e.g.,  schools, 
places  of  worship)  and  infrastructure  (e.g.  roads,  cell  towers)  are  indirect  indicators  of  social 
connectedness. 

Cultural,  social,  and  economic  factors  critical  to  understanding  societal  attitudes  are  associated 
with  specific  phenomena  observable  from  overhead  imagery.  Distinguishing  among  industrial, 
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commercial,  and  residential  areas,  for  example,  is  a  standard  use  for  imagery  (Harvey,  et  al.,  2002; 
Harvey,  McGlone,  McKeown,  &  Irvine,  2004;  Jenson  &  Cowan,  1999;  O'Brien  &  Irvine,  2004). 
Researchers  can  also  extract  measures  of  socioeconomic  status  (e.g.,  house  size,  crop  area  and 
crop  vigor,  presence  of  vehicles)  from  high-quality  imagery.  Other  factors  may  also  be  inferred 
from  indicators  derived  from  the  imagery  such  as  population  density,  access  to  improved  roads, 
distances  to  commercial  and  governmental  centers,  and  attributes  of  communities.  Patterns  that 
emerge  from  the  correlation  of  the  geospatial  analysis  with  survey  data  can  suggest  phenomena 
observable  in  imagery  that  serve  as  reasonable  surrogates  for  the  direct  measurements  of  public 
opinion  (Irvine,  Regan,  &  Lepanto,  2012;  Irvine,  Lepanto,  Regan,  &  Young,  2012).  Researchers  can 
apply  various  image  processing  techniques  to  extract  and  identify  specific  indicators  from  the 
imagery  (Table  3). 


Table  3.  Illustrative  imagery  observables  and  derived  indicators 


Feature  Class 

Observables 

Derived  Features 

Land  Cover 

Arable  land 

Area  under  cultivation 

Crop  health 

Level  of  commercial  agriculture 
Expected  food  supply 

Buildings 

Sizes  of  buildings 

Number  of  types  of  buildings 

Size  of  residential  buildings 

Presence  of  a  "large"  building 

Degree  of  urbanization  or 

industrialization 

Population  density  and 

distribution 

Average  size  of  residential 
buildings 

Indication  of  a  mosque,  school, 
or  community  building 

Lines  of  Communication 

Road  network  (paved  and 
unimproved) 

Other  transportation 

Numbers  and  types  of  vehicles 
Distance  from  major  commercial 

centers 

Level  of  communication 

Infrastructure  to  support  local 

commerce 

Access  to  transportation 
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Panchromatic  Multispectral 


Imagery  from  Digital  Globe 
Provided  by  NGA 

Figure  4.  Illustrative  panchromatic  and  multispectral  imagery  of  one  village  in  Afghanistan. 


Panchromatic  Image  Multispectral  Image  Pan-sharpened  Image 


Imagery  from  Digital  Globe 
Provided  by  NGA 


Figure  5.  Panchromatic  and  multispectral  images  chips  with  the  pan-sharpened  product. 


To  illustrate,  we  present  analysis  of  commercial  panchromatic  and  multispectral  satellite  imagery 
collected  over  northern  Afghanistan  (Figure  4).  The  panchromatic  imagery,  having  finer  spatial 
resolution,  can  reveal  details  about  the  shapes  of  objects.  The  multispectral  imagery  has  coarser 
resolution,  but  includes  data  in  four  spectral  regions  (red,  green,  blue,  and  near-infrared).  This 
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spectral  information  can  reveal  material  properties,  such  as  the  types  of  roads,  buildings,  crops, 
and  vegetation.  Panchromatic  sharpening  (Alparone,  Wald,  Chanussot,  Bruce,  &  Thomas,  2007;  Lee 
&  Lee,  2010)  can  combine  the  two  images  to  produce  a  fine-resolution  multispectral  image  (Figure 
5). 


Imagery  from  Digital  Globe 
Provided  by  NGA 


Figure  6.  Illustrative  features  evident  for  each  village. 


An  extensive  literature  addresses  methods  for  extracting  features  from  remotely  sensed  imagery 
(Campbell,  1987;  Lillesand  &  Kiefer,  1994).  Analysis  of  the  imagery  in  this  example  leveraged  both 
spatial  and  spectral  information  to  identify  features  of  interest.  The  specific  features  extracted 
from  the  imagery  include  estimates  of  the  number  of  buildings,  building  sizes,  building  density, 
extent,  and  health  of  crops,  and  types  of  roads  (Figure  6).  Derived  features  also  quantified  the  size, 
shape,  and  density  of  each  village.  To  supplement  the  automated  processing,  visual  inspection  of 
the  imagery  produced  ratings  of  relevant  characteristics.  Comparison  of  the  computed  features  to 
ground  truth  or  observer-generated  data  is  useful  for  assessing  the  performance  of  the  automated 
methods  (Irvine  et  al.,  2013). 

As  shown  in  Figure  7,  automated  processing  of  the  pan-sharpened  imagery  employed  a  mix  of 
spectral  and  spatial  attributes  to  extract  features  of  interest  (Irvine  et  al.,  2013).  The  four-band 
multispectral  data  offers  information  about  material  type.  Spectral  analysis  shows  that  the  material 
properties  of  the  buildings  differ  from  the  surrounding  spectral  signatures.  Using  the  spectral 
signature,  we  identified  regions  as  buildings,  crops,  trees,  or  "other."  Spatial  analysis  considered 
edge  features.  For  example,  the  villages  typically  consist  of  a  number  of  closely  spaced  buildings 
that  give  rise  to  many  edges  in  the  panchromatic  scene.  Regions  of  high  edge  density  correspond  to 
the  spatial  extent  of  the  village.  Merging  the  spatial  and  spectral  information  yields  detection  of 
the  individual  buildings.  From  these  detections,  we  compute  the  number  of  buildings,  building 
sizes,  and  building  density.  The  vegetation  and  crop  analysis  begins  with  identification  of  the  crop 
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land  based  on  spectral  features.  To  assess  crop  health,  we  use  the  computed  normalized  difference 
vegetation  index,  which  indicates  the  presence  and  health  of  vegetation  (Kriegler,  Malila,  Nalepka, 
&  Richardson,  1969). 


Figure  7.  Image  processing  to  identify  built-up  areas  comprising  a  village. 


Chishti  Sharif 


Sang  Takht 


Imagery  from  Digital  Globe 
Provided  by  NGA 


Figure  8.  Imagery  data  can  provide  reliable  estimates  of  key  sociocultural  dynamics. 


Imagery  of  one  town  in  Chishti  Sharif  shows  that  crops  are  robust  and  the  town  has  access  to  water 
and  good  road  transportation  (Figure  8).  By  contrast,  a  village  in  Sang  Takht  shows  a  dispersed 
building  infrastructure  and  lower  crop  vigor.  These  factors  indicate  the  economic  health  of  the  two 
villages.  When  compared  to  ground  observations,  the  imagery-based  analysis  accurately  predicts 
the  economic  health  of  the  villages.  If  the  crops  are  healthy  and  the  village  has  good  access  to 
other  resources,  the  likelihood  is  high  that  the  economic  conditions  are  relatively  strong. 
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2.4.  Sensor  Data:  Video  Imagery 

Many  recent  discussions  talk  about  "patterns  of  life"— the  normal  activities  of  an  individual  or 
group.  The  patterns  of  life  that  define  normal  activities  are  inherently  specific  to  a  society.  These 
patterns  depend  on  cultural  norms  of  behavior,  economic  structure  and  incentives,  and  the 
physical  environment.  For  example,  an  activity  of  interest  could  pertain  to  the  behavior  of  people 
in  a  commercial  area,  such  as  an  open-air  market.  Is  it  normal  for  people  to  shop  in  groups  or 
individually?  Do  people  linger  and  socialize?  Difference  in  gender  roles  (Jain,  1985)  and  age 
structure  of  the  population  (Baltes,  Wahl,  &  Schmid-Furstoss,  1990)  affect  the  behavioral  norms. 

If  it  were  possible  to  model  normal  activity,  could  potential  threats  or  concerns  be  discovered  as 
departures  from  normalcy?  Analysis  of  activity  requires  temporal  understanding.  The  static 
imagery  discussed  in  the  previous  section  captures  a  snapshot  in  time,  but  understanding  more 
dynamic  phenomena  requires  repeated  observations  over  time— a  process  to  which  video  imagery 
is  ideally  suited. 

Video  data  provides  the  capability  to  analyze  temporal  events,  which  enables  far  deeper  analysis 
than  is  possible  with  still  imagery.  At  the  primitive  level,  analysis  of  still  imagery  depends  on  the 
static  detection,  recognition,  and  characterization  of  objects,  such  as  people  or  vehicles.  By  adding 
the  temporal  dimension,  video  data  reveals  information  about  the  movement  of  objects,  including 
changes  in  pose  and  position  and  changes  in  the  spatial  configuration  of  objects.  This  additional 
information  can  support  the  recognition  of  basic  activities,  associations  among  objects,  and 
analysis  of  complex  behavior. 

The  information  complexity  associated  with  analysis  of  a  target  object  has  a  natural  hierarchy.  The 
progressive  stages  of  processing  appear  in  Figure  9,  where  successive  components  indicate  the 
decreasing  maturity  of  current  methods.  The  first  two  components  on  the  left  exploit  information 
in  the  sensor  phenomenology  domain,  while  the  two  components  on  the  right  exploit  extracted 
features  derived  from  the  sensor  data. 


Observed  Derived 


Figure  9.  Levels  of  exploitation  for  motion  imagery  data. 
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To  illustrate  the  concept,  consider  a  security  application  with  a  surveillance  camera  overlooking  a 
bank  parking  lot.  If  the  bank  is  robbed,  a  camera  that  collects  still  images  might  acquire  an  image 
depicting  the  robbers  exiting  the  building  and  show  several  cars  in  the  parking  lot.  The  perpetrators 
have  been  detected  (stage  1  in  the  figure),  but  additional  information  is  limited.  A  video  camera 
might  collect  a  clip  showing  the  same  people  entering  a  specific  vehicle  for  their  getaway.  Now 
both  the  perpetrators  and  the  vehicle  have  been  identified  because  the  activity  (a  getaway)  was 
observed.  This  is  the  second  stage  of  understanding,  which  requires  acquisition  of  temporal 
information  to  recognize  an  activity.  Moving  to  stage  3,  we  consider  relationships  among 
information  acquired  in  multiple  times  and  places.  If  other  security  cameras  throughout  the  city 
detect  the  vehicle  in  our  example,  analysis  of  multiple  videos  could  reveal  the  pattern  of 
movement  and  suggest  the  location  of  the  robbers'  base  of  operations.  In  this  way,  observers  can 
form  an  association  between  the  event  and  specific  locations,  namely  the  bank  and  the  robbers' 
hideout.  If  the  same  perpetrators  were  observed  over  several  bank  robberies,  one  could  discern 
their  pattern  of  behavior,  i.e.,  their  modus  operandi.  Thus,  the  fourth  and  final  level  of  processing 
examines  a  corpus  of  data  to  assess  patterns  of  activities  and  associations.  This  information  could 
enable  law  enforcement  to  anticipate  future  events  and  respond  appropriately  (Gualdi,  Prati,  & 
Cucchiara,  2008;  Porter,  Fraser,  &  Hush,  2010). 

Methods  for  recognizing  activities  in  video  data  encompass  a  multistep  process  that  begins  with 
tracking  objects,  such  as  people  or  vehicles,  in  the  video  data  (McPherson,  Irvine,  Young,  & 
Stefanidis,  2012).  By  analyzing  the  speeds  and  trajectories  of  these  objects  and  their  relations  to 
other  objects,  we  can  infer  some  understanding  of  behaviors.  For  example,  one  person  stationary 
on  a  sidewalk  may  be  loitering,  but  another  person  approaching  him  represents  a  meeting. 
Researchers  have  developed  methods  for  analyzing  the  kinematics  to  infer  primitive  activities  and 
construct  an  understanding  of  complex  activities. 

Several  standard  methods  can  perform  the  first  stage  of  processing— converting  raw  video  frames 
into  object  tracks  (Beymer,  McLaughlin,  Coifman,  &  Malik,  1997;  Regazzoni,  Fabri,  &  Vernazza, 
1999).  This  video  conversion  step  is  the  only  process  that  references  the  raw  video  directly. 
Subsequent  processing  depends  on  the  object  positions  and  velocities  (i.e.,  the  kinematics).  This 
approach  is  sensor  agnostic  in  that  multiple  sensors  could  provide  suitable  object  track  data  and 
the  activity  analysis  can  combine  and  compare  information  across  the  sensor  types.  For  example, 
information  from  full  motion  video  could  be  combined  with  information  derived  from  wide-area 
motion  imagery  or  ground  moving  target  indicator  data.  The  fundamental  requirement  is  that  the 
data  source  has  sufficient  data  quality  to  support  accurate  information  extraction  (McPherson  et 
al.,  2012;  Wood,  McPherson,  &  Irvine,  2012).  Simple  activities  include  a  person  walking,  running, 
standing  close  to  a  car,  or  approaching  another  person,  vehicles  stopping  at  particular  locations, 
etc.  (Table  4).  Complex  activities  are  linearly  ordered  time  sequences  of  simple  behaviors  (e.g., 
persons  interacting  by  conversing  or  exchanging  a  package,  and  vehicles  traveling  in  convoy  or 
making  a  rendezvous)  (Figure  10). 
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Table  4.  Processing  levels  for  representation  of  complex  activities 


Semantic  Level  of 

Activity 

Example  of  Activities 

Analytic  Methods 

Level  1  activities 

(atomic  level) 

Meet,  enter,  exit,  loiter 

Tokenization 

Level  2  activities 

(interaction  of  tracks) 

Attraction  (location  where  moving  objects 
pause,  before  continuing) 

Pick-up  /  drop-off 

Mount/ dismount  (enter/exit  mover) 

Portal  (enter  stationary  object) 

Recognition  by  syntactic 

methods 

Level  3  activities 

(interaction  of  level  1 
and  2  activities) 

Object  hand-off  (drop-off,  pick-up) 

Tribal  elder  (many  meetings  by  relatively 
stationary  person) 

Bury  IED  (exit  vehicle,  move,  loiter,  move, 
enter  vehicle) 

Recognition  by  syntactic 

methods 

Kinematics 


•  Object  velocities 

•  Inter-object  distances 


*  Group  primitives 

-  Membership  sizes 

-  Cluster  locations 

-  Cluster  velocities 

-  Distances 


Are  simple  activities 

-  Move/stop 

-  Meet/disperse 

-  Merge/split 

-  Appear/disappear 


» 


Syntactic 


¥ 


High-level 

Activity 

Identification 


•  Higher  level  tokens 
Are  more  complex 
Activities 

-  Mob  scene 

-  Bank  robbery 


Figure  10.  Overview  of  syntactic  data  processing  method. 


Once  object  positions  and  tracks  are  extracted  from  the  video  data,  the  second  layer  of  processing 
tokenizes  the  track  data  (i.e.,  converts  short  increments  of  track  data  into  a  symbolic 
representation)  according  to  a  finite  set  of  tokens  based  on  position,  velocity,  and  time.  This 
tokenization  step  yields  a  compact,  symbolic  representation  of  information  on  entities  in  a  form 
appropriate  for  syntactic  recognition.  Researchers  have  explored  different  techniques  for  this 
tokenization.  One  approach  is  a  geometrically  invariant  kinematic  analysis,  which  employs 
heuristics  to  characterize  tracks  in  space  and  time  and  the  relationships  among  entities  in  space- 
time.  An  alternative  approach  applies  a  clustering  method  to  the  track  data  to  discover  atomic 
elements.  Subsequent  analysis  rests  on  syntactic  methods  for  string  representations  of  these 
elements. 
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In  general,  real-world  activities  are  semantically  complex.  Recognizing  behaviors  requires  the 
ability  to  build  discriminative  models  that  generalize  across  missions.  Representing  simple 
behaviors  using  symbols  or  "tokens/'  as  described  above,  facilitates  pattern  discovery.  The  time 
sequence  of  simple  activities  is  represented  as  a  string  of  symbols.  Researchers  can  discover  the 
recurring  patterns  in  simple  behaviors  through  linguistic  processing  techniques  that  operate  on  the 
symbol  strings.  The  symbols  constitute  the  "alphabet"  for  encoding  activity  patterns.  Frequent 
patterns  constitute  normalcy.  Statistically  improbable  or  unique  patterns  would  be  flagged  as 
anomalies,  i.e.,  possible  activities  of  interest. 

When  the  underlying  behavior  sequence  contains  noise  and  stochastic  variation  (which  is  common 
for  activities  involving  humans),  "tolerant"  pattern  discovery  that  finds  approximate  matches  is 
required  to  ensure  detection  of  all  activities.  The  Fast  Approximate  Sub-tree  Matching  (FASM) 
technique  (Deutsch,  Kolacinski,  &  Peli,  2010)  generates  a  prefix  tree  from  the  symbol  stream, 
converts  this  structure  to  graph-invariant  canonical  strings,  and  finds  all  patterns  and  their 
frequency.  The  number  of  nodes  is  a  function  of  the  size  of  the  symbol  alphabet,  the  number  of 
distinct  activity  patterns,  and  the  maximum  pattern  size  under  investigation.  FASM  discovers 
approximate  matches  by  calculating  the  edit  distance4  between  canonical  strings  and  by  exploiting 
the  bounding  properties  of  the  edit  distance  to  accelerate  the  process. 

Anthropologists  learn  about  the  culture  of  another  society  through  first-hand  observation  in  that 
society,  i.e.,  ethnographic  studies.  Because  culture  primarily  relates  to  the  way  people  interact, 
researchers  cannot  observe  it  adequately  in  a  laboratory  setting.  The  systematic  observation  of 
activities  from  video  imagery  provides  a  limited  type  of  ethnographic  glimpse  into  a  society.  The 
movement  and  associations  of  people  and  vehicles  reveal  normal  patterns  of  activity  for  the 
society.  A  subject  matter  expert  in  a  particular  society  could  suggest  patterns  that  analysts  could 
anticipate.  By  processing  and  analyzing  sensor  data  collected  over  a  particular  region,  the  patterns 
that  comprise  the  cultural  model  can  be  learned  over  time. 

Cultural  differences  in  normal  activities  can  be  derived  from  a  comparison  of  models  for  two 
cultures.  Thus,  within  the  framework  of  activities  defined  by  symbolic  representations  of 
movements  of  people  and  vehicles,  a  comparison  of  two  normalcy  models  provides  a  type  of 
ethnology  for  understanding  the  differences  in  behavior  between  two  cultures.  As  an  example,  for 
the  counterinsurgency  mission  this  type  of  analysis  could  reveal  the  differences  in  behavior 
between,  say,  Iraq  and  Afghanistan.  The  model  developed  for  one  region  (and  its  ability  to  detect 
anomalous  behavior)  may  not  translate  immediately  to  a  new  region. 


4  The  edit  distance  (also  called  the  Levenshtein  distance)  between  two  strings  of  characters  is  the  minimum 
number  of  single-character  edits  (insertion,  deletion,  substitution)  required  to  change  one  string  into  the 
other. 
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3.  Inference  and  Analysis 

Numerous  issues  arise  in  the  analysis  of  the  various  information  sources  discussed  in  the  previous 
section.  In  many  cases,  standard  methods  exist  for  conducting  the  appropriate  analysis.  For 
example,  many  books  have  been  written  about  statistical  methods,  pattern  recognition,  data 
mining,  natural  language  processing,  and  related  topics.  However,  several  new  themes  have 
emerged  in  recent  years  that  merit  greater  attention.  Two  themes  we  address  here  are  the  use  of 
network  modeling  and  the  detection  of  changes  over  time.  Sociocultural  analysis  must  recognize 
that  the  actors  are  individuals  and  groups  that  have  relations  with  other  actors.  Graph 
representations  of  these  relationships  provide  a  powerful  set  of  mathematical  tools  for 
understanding  and  modeling  these  relationships.  Analytic  methods  for  change  detection  provide  a 
method  for  discerning  when  and  how  events  and  relationships  shift  over  time. 

3.1.  Discovering  and  Analyzing  Relationships 

Social  network  models  offer  a  rich  mathematical  framework  for  analyzing  interactions  among 
agents,  whether  individuals  and/or  groups  of  individuals.  In  a  social  network  model,  agents  are 
represented  by  nodes  in  a  graph  and  the  relationships  among  agents  by  edges  (arcs)  in  the  graph. 
Depending  on  the  application,  the  agents  could  be  single  individuals  or  specific  groups  of  people 
that  can  be  modeled  as  a  single  entity.  In  some  cases,  nodes  may  represent  locations  of  interest, 
rather  than  actors,  and  the  edges  may  represent  routes  connecting  these  locations. 

Many  sources  of  data  can  be  used  to  discover  or  estimate  the  elements  and  structure  of  the 
network.  Research  has  demonstrated  construction  of  social  networks  from  text  (Carley,  1997)  and 
social  media  (Carley,  2002).  Multiple  tools  have  been  developed  for  analysis  of  networks 
(D'Andrea,  2010;  Freeman,  2006;  Newman,  Barabasi,  &  Watts,  2006;  Pfeffer  &  Carley,  2012; 
Wasserman  &  Faust,  1994)  and  new  tools  are  being  applied  specifically  to  support  military  missions 
(Mathieu,  Lorber,  Ounanian,  Fulk,  Troop,  &  Bornmann,  2012).  The  field  is  far  too  extensive  to 
describe  here;  rather,  we  explore  two  practical  issues  in  the  analysis  of  networks. 

3.1.1.  Incomplete  view  of  the  network 

Ideally,  it  would  be  possible  to  observe  all  of  the  nodes  and  edges  and  achieve  a  complete 
understanding  of  the  network.  In  practice,  researchers  must  usually  construct  inferences  from 
limited  observations.  One  natural  topic  to  consider  is  the  completeness  of  the  observations  of  the 
network.  Do  the  available  data  represent  most  of  the  network  or  are  we  only  seeing  the  tip  of  the 
iceberg? 

We  illustrate  this  problem  using  synthetic  data  that  represent  a  small  network  aimed  at  the 
production  and  emplacement  of  improvised  explosive  devices  (lEDs).  A  limited  picture  of  such  a 
network  could  emerge  from  analysis  of  data  on  the  movements  of  network  members  drawn  from 
motion  imagery  or  other  data  sources. 

Consider  a  simplified,  hypothetical  example  application  of  network  analytics,  in  which  a  small  IED 
network  consists  of  15  nodes,  corresponding  to  locations  involved  in  the  planning,  manufacture, 
and  execution  of  IED  attacks  (Figure  11).  This  network  has  three  sub-networks  corresponding  to  the 
planning,  manufacturing,  and  execution  aspects  of  the  IED  process.  Each  sub-network  shows  high 
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internal  connectivity  but  limited  connection  to  the  other  sub-networks.  Analysis  of  a  large  set  of 
track  data  derived  from  motion  imagery  could,  in  principle,  reveal  the  network  by  observing  the 
connections  among  the  locations  associated  with  each  node.  Figure  11,  Panel  B  represents  the  true 
network  as  a  directed  graph.  A  Monte  Carlo  simulation  of  the  tracks  in  this  network  generated 
1,000  tracks  that  yield  the  estimated  network  in  Figure  11,  Panel  C.  The  small  difference  between 
the  true  and  estimated  networks  indicates  a  good  estimate  of  the  network.  In  reality,  inferences 
about  the  network  would  be  subject  to  noisy  data,  spurious  nodes  and  links,  limited  observations, 
and  changes  in  networks  over  time. 


Figure  11.  Simple  example  of  an  IED  network.  Panel  A  shows  the  simplified  network,  consisting  of 
three  sub-networks.  Panel  B  is  a  graph-based  representation,  where  nodes  correspond  to  rows  and 
columns,  and  the  height  of  the  bar  is  the  strength  of  the  edge  between  the  two.  Panel  C  is  an 
estimate  of  the  graph  from  a  large  sample  of  observations. 


Network  fidelity  analysis  uses  computer-intensive  capture-recapture  methods  to  assess  how  well 
the  current  model  represents  the  actual  network.  Limited  data  may  give  a  very  incomplete  picture 
of  the  true  network.  Therefore,  the  method  is  based  on  a  statistical  technique  called  capture- 
recapture,  which  estimates  how  much  of  the  network  has  been  observed.  The  technique  uses  two 
samples  of  network  data.  Capture-recapture  analysis  (Cormack,  1989)  compares  the  nodes  that 
arise  in  both  samples  to  the  nodes  that  appear  in  only  one  sample  to  estimate  the  number  of 
nodes  that  appeared  in  neither  sample,  i.e.,  the  unobserved  portion  of  the  network.  In  our  simple 
example  (Figure  12),  even  though  only  12  nodes  appear  in  at  least  one  of  the  samples,  and  only  8 
nodes  appear  in  both,  the  estimated  total  number  of  nodes  is  14.67,  remarkably  close  to  the  true 
value  of  15. 
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Sample  1 


Sample  2 


Figure  12.  Two  estimates  of  the  directed  graph  from  Figure  11,  each  estimated  from  10  tracks. 
Note  that  the  two  independent  estimates  reveal  substantially  different  views  of  the  network. 


In  practice,  the  two  samples  can  be  constructed  by  partitioning  a  large  data  set.  For  a  given  set  of 
data,  the  partition  into  two  "samples"  can  be  arbitrary.  A  computer-intensive  resampling  method 
for  randomly  partitioning  the  data  and  repeating  the  capture-recapture  analysis  can  be  used  to 
construct  both  the  point  estimate  and  a  confidence  interval.  Applying  this  technique  to  our  simple 
example  yields  a  final  estimate  of  14.89  nodes  and  a  95%  confidence  interval  of  13.7  to  16.1.  This 
technique  can  also  be  applied  to  the  edges  of  the  graph  to  assess  our  understanding  of  the 
relationships  within  the  network.  Additional  discussion  of  capture-recapture  methods  appear  in  the 
text  box. 
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Capture-Recapture  Methods 

Capture-recapture  is  a  statistical  technique  originally  developed  for  estimating 
populations  in  the  wild.  Imagine  that  you  wish  to  estimate  the  number  of  fish  in  a 
pond.  Suppose  you  capture  100  fish,  tag  them,  and  release  them.  A  short  time  later, 
you  capture  another  100  fish.  Suppose  10  of  the  fish  in  this  second  set  had  tags. 
Because  10  out  of  100  fish  were  tagged  in  the  second  sample,  we  estimate  that 
tagged  fish  represent  approximately  10  percent  of  the  population.  But  we  know  the 
total  number  of  tagged  fish,  because  that  was  the  first  sample  of  100  fish.  Hence,  we 
estimate  the  total  population  to  be  approximately  1,000. 

The  method  derives  its  name  from  the  two-stage  sampling  process.  Stage  one  (the 
"capture")  determines  the  tagged  population  and  stage  two  (the  "recapture") 
assesses  the  proportion  of  the  total  population  tagged.  The  techniques  have  been 
widely  used  in  wildlife  and  ecology  studies.  More  recently,  the  methods  have  been 
applied  to  estimating  prevalence  of  disease. 

To  demonstrate  the  details,  consider  the  two  samples  as  portrayed  in  the  table  below, 
Ni  is  the  total  observed  from  sample  1  and  N2  is  the  total  from  sample  2.  We  denote 
the  number  common  to  both  by  n  1,1.  Hence  ni,ois  the  number  in  sample  1  that  were 
not  in  sample  2;  similarly,  no,i  is  the  number  in  sample  2  that  were  not  in  sample  1. 

The  number  not  observed  in  any  sample  is  no,o,  and  this  is  the  value  we  need  to  infer. 


Sample  2 

Observed 

Not  Observed 

Total 

Sample  1 

Observed 

fll,l 

ni,o 

Ni 

Not  Observed 

fio,i 

no,o 

TOTAL 

n2 

N 

The  capture-recapture  estimate  for  the  total  population  is: 

N  =  (ni,i+  ni,0)  (fh, i+  no,i)  /  rh, i  =  (Ni)(N2)  /  rh,i 
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Capture-Recapture  Methods  continued 

and  the  number  of  unobserved  elements  is: 

no,o  —  N  —  A/ 1  —  A/2  +  Hi,i 


The  capture-recapture  methods  rely  on  simple  principles  of  sampling.  The  method 
assumes  that  both  samples  were  drawn  randomly  from  the  population.  The  method 
also  assumes  an  open  mixing  of  the  population,  such  that  the  proportion  of  tagged 
elements  in  the  second  sample  indicates  the  proportion  in  the  population.  For  some 
practical  applications,  the  sampling  methods  may  depend  more  on  the  methods  for 
acquiring  the  data.  In  the  network  application,  different  sources  of  information  may 
sample  certain  regions  disproportionately,  leading  to  systematic  differences  in  the 
sampling  processes.  In  this  case,  extensions  of  capture-recapture  to  stratified  samples 
and  other  complex  sampling  strategies  has  been  an  active  area  of  research. 

Furthermore,  we  assume  that  elements  common  to  both  samples  can  be  easily 
identified.  Again,  this  can  be  a  problem  for  some  applications.  Suppose  that  one  were 
studying  a  network  formed  by  people  contacting  each  other  through  phone  calls.  A  call 
from  one  phone  to  another  provides  evidence  of  an  edge  linking  the  two  nodes.  If  one 
source  of  data  identified  the  nodes  by  the  phone  number  and  another  identified  them 
by  name,  recognizing  which  nodes  were  common  to  the  two  lists  would  require 
additional  information. 


3.1.2.  Network  modeling  influence 

Social  network  models  indicate  the  connections  and  relationships  among  the  actors.  Simple  models 
capture  the  relationship  between  two  nodes  as  an  edge  or  link  that  connects  them.  More  complex 
relationships  can  be  portrayed  by  assigning  attributes  to  these  links,  including  directionality.  A 
network  model  of  an  organization,  for  example,  could  represent  the  authority  structure  as  a 
directed  graph  to  indicate  a  supervisory  relationship  between  two  nodes.  A  similar  approach  can 
be  used  to  model  the  propagation  of  influence  within  a  network. 

We  present  a  simple  social  network  influence  model  to  illustrate  the  possibilities.  The  starting  point 
for  this  is  a  Markovian  model  developed  by  Acemoglu  (Acemoglu  &  Parandeh,  2009;  Acemoglu  & 
Ozdaglar,  2010;  Acemoglu,  Como,  Fagnani,  &  Ozdaglar,  2013;  Acemoglu,  Bimpikis,  &  Ozdaglar, 
2014).  The  model  represents  the  agents  as  nodes  in  the  network  graph.  Values  at  each  node 
indicate  the  attitudes  of  the  respective  agent  and  the  edges  of  the  graph  represent  connections 
among  the  agents  over  which  influence  occurs.  The  level  of  influence  an  agent  (node)  exerts  on 
other  agents  (nodes)  is  modeled  by  the  degree  of  forcefulness  of  each.  The  technique  has  been 
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applied  to  modeling  influence  in  a  village  society,  such  as  rural  Afghanistan  (Hung,  Kolitz,  & 
Ozdaglar,  2011;  Hung  &  Kolitz,  2011;  Hung,  Kolitz,  &  Ozdaglar,  2013).  Figure  13  illustrates  a  simple 
example  of  three  villages,  with  the  color  of  a  node  representing  the  value;  i.e.,  a  measure  of  the 
attitude  of  the  agent,  where  0.5  is  "good/Blue,"  i.e.,  most  positive  attitude  toward  the  Coalition 
Forces  and  -0.5  is  "bad/Red,"  i.e.,  most  positive  toward  the  Taliban.  The  consensus  of  the  network 
at  a  point  in  time  is  the  mean  of  all  node  values.  In  the  absence  of  exogenous  influence,  a  steady- 
state  consensus  will  be  achieved. 
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Figure  13.  Illustrative  network  model.  The  color  of  each  node  indicates  value,  i.e.,  the  measure  of 
the  agent's  current  attitude. 


The  state  of  the  system  is  the  set  of  values  for  the  nodes  in  the  network.  Changes  in  the  state  of 
the  system,  i.e.,  transitions,  occur  when  two  nodes  interact.  The  transition  resulting  from  an 
interaction  depends  upon  the  forcefulness  of  the  agents.  In  the  interaction  between  two  agents 
the  possible  outcomes  are  (1)  no  change  in  values,  (2)  one  agent's  value  moves  closer  to  the 
other's,  or  (3)  both  agents'  values  move  closer  together.  The  model  treats  each  of  these  outcomes 
as  occurring  with  some  probability  that  depends  on  the  forcefulness  of  the  two  agents.  In  practice, 
the  forcefulness  depends  on  each  agent's  position  in  the  society  and  personality  factors  that  may 
be  difficult  to  observe  and  quantify.  In  an  exchange  between  two  agents,  the  agent  most  likely  to 
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change  in  value  is  the  less  forceful  of  the  two  and  the  amount  of  change  depends  on  a  parameter 
that  measures  consistency  of  attitude.  Furthermore,  change  does  not  always  occur.  The  three 
possibilities  are:  (1)  agents  i  and  j  reach  a  pair-wise  consensus;  (2)  agent  j  exerts  influence  on  agent 
i,  where  the  parameter  si  measures  the  tendency  for  agent  i  to  resist  change  to  his/her  attitude  in 
response  to  the  interaction;  and  (3)  agents  i  and  j  do  not  agree  and  both  retain  their  original 
attitudes. 

The  model  has  been  instantiated  in  a  Monte  Carlo  simulation,  which  facilitates  investigation  of 
alternative  methods  for  introducing  influence.  For  example,  will  a  particular  influence  have  a 
greater  effect  on  the  overall  consensus  if  it  is  first  introduced  to  a  forceful  agent  (i.e.,  someone  in  a 
position  of  authority)  or  at  a  lower  level  in  the  social  hierarchy?  The  model  described  above 
focuses  on  the  steady-state  behavior  of  the  network,  but  new  investigations  are  exploring  the 
dynamic  behavior  where  opposing  points  of  view  compete  for  influence. 

3.2.  Change  Detection 

Identifying  a  shift  in  the  normal  pattern  of  activity  or  behavior  can  alert  observers  to  important 
events.  Shifts  in  the  content  or  sentiment  of  social  media  sites  could  indicate  a  rise  in  political 
discontent,  possibly  as  a  prelude  to  civil  unrest.  Changes  in  the  pattern  of  online  searches  about 
common  medical  conditions  could  suggest  an  outbreak  of  disease.  Differences  in  consumer 
spending  after  implementation  of  a  new  economic  policy  could  indicate  the  effects  of  the  policy.  In 
all  of  these  cases,  however,  some  level  of  random  fluctuations  is  expected  and  normal.  The 
methodological  challenge  is  to  detect  the  changes  that  are,  in  some  sense,  "real/'  In  this  section, 
we  briefly  examine  some  methods  for  detecting  changes  and  illustrate  the  process  with  a  simple 
example. 

A  fundamental  difference  exists  between  two  types  of  change  detection  problems  (Irvine,  1982).  In 
one  case,  we  know  the  time  of  the  possible  change  and  want  to  know  if  a  shift  really  did  take  place. 
In  the  second  case,  we  continuously  monitor  a  process  or  condition  and  want  to  detect  a  change  at 
any  point  in  time.  In  the  above  examples,  the  assessment  of  the  new  economic  policy  is  an  instance 
of  the  first  type  of  change.  We  know  when  the  new  economic  policy  was  implemented  and  we 
want  to  assess  its  effects.  The  examples  of  political  unrest  and  disease  outbreak  fall  into  the  second 
type  of  change  detection.  The  events  could  occur  at  any  point  in  time  and  only  continuous 
monitoring  can  detect  them. 

For  the  first  type  of  change  detection,  standard  statistical  methods  can  address  many  of  the 
problems  that  arise  in  practice.  If  researchers  have  access  to  data  both  before  and  after  the 
relevant  point  in  time,  they  can  apply  standard  techniques  to  test  for  a  difference  between  the  two 
time  periods.  The  second  type  of  analysis— detecting  the  change  at  an  unknown  point  in  time- 
poses  the  tricky  analytic  problem  (Basseville  &  Nikiforov,  1993;  Brodsky  &  Darkhovsky,  1993;  Chen 
&  Gupta,  2000).  Rather  than  present  the  mathematical  treatment  of  this  issue,  we  illustrate  the 
practical  implications  with  an  example. 

Consider  a  sequence  of  observations  collected  over  time.  In  the  spirit  of  our  first  example,  these 
observations  might  show  the  percentage  of  posts  on  a  blog  that  exhibit  discontent  with  the 
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government.  We  expect  some  fluctuations  in  this  percentage  as  various  events  occur  and  news 
stories  are  reported.  Using  synthetic  data,  we  present  two  scenarios:  one  with  no  change  and  one 
where  a  real  change  occurs  (Figure  14).  Note  the  random  fluctuations  in  the  two  data  series. 


The  challenge  is  to  detect  the  true  change  without  being  deceived  by  the  random  fluctuations. 
First,  suppose  an  analyst  mistakenly  thinks  that  standard  statistical  methods  can  be  applied  to  this 
type  of  problem.  We  present  this  case  because  it  illustrates  the  fundamental  difference  between 
the  two  types  of  change  detection  problems  defined  above.  A  naive  analyst  might  try  to  apply  a 
standard  statistical  technique.  Suppose  one  used  the  Student's  t-statistic  to  look  for  a  change  at 
each  point  in  the  series  (Figure  15).  This  analysis  erroneously  indicates  numerous  changes  when  in 
fact  no  change  has  occurred.  These  incorrect  changes  or  false  alarms  will  mislead  all  subsequent 
analysis. 
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Now  we  repeat  the  analysis  with  an  appropriate  test  procedure.  Statistical  methods  to  handle  the 
detection  of  a  change  at  an  unknown  point  in  time  apply  to  activities  such  as  quality  control  for 
manufacturing,  target  tracking,  and  biomedical  research.  The  common  element  in  many  of  these 
techniques  is  that  they  operate  sequentially  on  the  data  to  detect  changes  as  soon  as  possible  after 
they  occur.  Another  class  of  methods  has  been  developed  for  retrospective  or  forensic  analysis  of 
change  points.  Searching  for  a  change  in  the  series  involves  an  inherent  trade-off  between  missed 
detections  (failure  to  detect  a  true  change)  and  false  alarms  (alerting  when  no  change  has  actually 
occurred).  The  statistical  procedures  have  thresholds  that  can  be  set  according  to  the  acceptable 
risks  for  these  two  types  of  errors.  In  our  synthetic  example,  the  choice  of  thresholds  led  to  one 
false  alarm,  but  accurate  detection  of  the  change  (Figure  16). 
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•  Change 


Figure  16.  Applying  an  appropriate  test  procedure  yields  far  better  results. 


A  related  problem  affects  assessments  of  social  network  stability.  Changes  in  the  network  over 
time  could  indicate  a  response  to  action  by  exogenous  influences  or  shifts  in  behavior  for  other 
reasons.  Noise  in  the  observation  of  the  network  will  produce  random  fluctuations  that  might  look 
like  changes  when  no  true  change  has  occurred.  Random  fluctuations  in  the  process  can  mislead 
techniques  that  "detect"  change  by  using  heuristics  or  by  observing  that  something  is  "different" 
from  before.  Similarly,  as  we  saw  above,  naively  applying  a  standard  statistical  test  will  lead  to 
incorrect  decisions. 

Network  stability  analysis  uses  statistical  methods  to  detect  significant  change  points  in  the  graph- 
based  representation  of  the  network,  while  guarding  against  incorrect  use  by  a  naive  user. 
Extending  statistical  change  point  methods  to  graph-based  random  variables  enables  a  useful 
approach  to  network  stability  analysis.  In  our  simple  example,  Figure  18  shows  metrics  computed 
from  estimates  of  the  network  over  successive  time  periods.  For  simplicity,  the  example  shows  a 
series  of  scalar  values,  but  the  appropriate  metric  could  be  a  vector  representing  a  set  of  statistics 
or  the  actual  estimates  of  the  directed  graph.  Applying  a  cumulative  sum  (CUSUM)  technique  (van 
Dobben  de  Bruyn,  1968;  Wu,  2005)  reveals  the  correct  change  point  (Figure  17).  Additional 
discussion  of  CUSUM  techniques  appears  in  the  text  box. 
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Figure  17.  Notional  depiction  of  change  detection.  Upper  plots  (a)  show  a  case  with  no  change  and 
lower  plots  (b)  show  a  significant  change.  The  CUSUM  technique  correctly  indicates  when  and 
where  significant  changes  occur. 


CUSUM  Tests 


The  CUSUM  test  is  a  statistical  procedure  for  detecting  a  structural  change  in  a  model 
for  time  series  data.  The  method  is  a  special  application  of  a  sequential  procedure  that 
has  been  widely  applied  in  quality  control  for  manufacturing  processes  and  to  model 
economic  data.  The  objective  of  the  technique  is  to  detect  a  change  in  an  underlying 
process  as  early  as  possible. 
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CUSUM  Tests  continued 

For  example,  if  {X(f)|  represents  the  average  price  of  cotton  clothing  in  a  particular 
country,  one  could  model  this  as  a  function  of  various  factors  that  influence  prices. 
The  model  for  0  could  depend  on  labor  costs  for  workers  in  the  garment  industry,  the 
price  of  raw  cotton,  seasonal  factors  that  influence  demand,  and  possibly  other 
variables. 

A  change  in  the  model  would  indicate  a  shift  in  the  structure  of  the  economic  factors 
that  drive  the  prices. 

The  motivation  for  the  technique  is  that  the  time  series  can  be  modeled  by 

X(t)  =  0  +  e(t) 

where  {  e(t) }  represent  independent  and  identically  distributed  errors  with  mean  zero 
and  finite  variance.  Often  e(f)  is  also  assumed  to  be  Gaussian.  Under  these  conditions, 
X(t)  -  0  represents  the  observed  error  in  the  model. 


As  long  as  0  is  the  appropriate  model  the  sum  of  these  deviations  has  expectation  zero 
and  a  variance  determined  by  the  variance  of  s (t)  and  the  number  of  observations.  In 
general,  the  statistic 

T 

5(7)  =  ^V(t)  -  0] 

t  =  1 

is  a  Gaussian  process  with  expectation  zero  and  a  variance  that  grows  over  time.  The 
CUSUM  test  compares  5(7")  to  limits  defined  by  the  variance  and  the  levels  of 
acceptable  type  1  and  type  2  error.  When  the  statistic  5(7")  wanders  beyond  these 
limits,  we  judge  that  a  change  has  occurred. 


4.  Technology  Gaps 

Methods  for  processing  and  analysis  of  many  data  sources  have  equipped  researchers  with  new 
tools  for  sociocultural  investigations.  However,  many  challenges  remain.  Some  of  the  key  gaps  in 
current  technology  span  the  full  range  from  data  collection  to  data  management  and  analysis.  As 
newer  technologies  address  these  challenges,  methods  available  for  processing  and  analyzing  data 
will  become  appropriate  not  just  for  researchers  but  also  for  wide  operational  use. 


142  |  Transforming  data  into  information 


4.1.  Data  Collection 

Researchers  need  better  methods  for  defining  data  needs  and  understanding  the  nature  of  the 
data  collected.  In  particular,  while  social  media  are  emerging  as  a  data  source  of  interest  that 
provide  a  tremendous  amount  of  information,  they  raise  a  serious  concern  with  regard  to 
understanding  how  findings  based  on  social  media  might  generalize  to  the  broader  population. 
Unlike  traditional  survey  research,  we  lack  good  sampling  models  for  exploring  social  media.  Many 
younger,  technology-savvy  people  have  embraced  this  new  form  of  communication,  but  we  need 
better  understanding  of  how  these  people  differ  from  those  not  using  these  channels  of 
communication. 

A  related  gap  arises  in  learning  how  to  collect  data  from  traditional  sources  for  new  applications. 
Remote  sensing  and  imagery  analysis  have  been  applied  successfully  to  a  number  of  earth  science 
and  intelligence  needs,  but  are  just  emerging  as  tools  for  social  science  research.  For  researchers  to 
exploit  imagery  in  this  new  context,  the  imagery  quality,  spectral  bands,  and  frequency  of 
collection  must  be  adequate  to  support  the  analysis.  Traditional  methods  for  specifying  imagery 
collection  needs  used  technical  parameters,  such  as  the  ground  sample  distance  of  the  imagery,  or 
functional  requirements  expressed  using  the  National  Imagery  Interpretability  Rating  Scales  (NIIRS) 
(Irvine,  2003;  Leachtenaur  &  Driggers,  2001;).  The  research  community  has  not  yet  developed  the 
framework  for  expressing  imagery  collection  needs  for  social  science  applications.  The 
development  of  the  Civil  NIIRS  (Hothem,  1996)  represented  a  step  in  this  direction,  but  much  work 
remains. 

4.2.  Data  Management 

The  volume  of  data  available  today  is  overwhelming.  Sociocultural  analysis,  like  other  disciplines, 
will  benefit  from  the  revolution  in  Big  Data.  In  particular,  new  data  storage  and  management 
technology  will  make  it  possible  to  perform  analyses  that  were  inconceivable  a  few  years  ago.  New 
methods  for  handling  data,  including  data  representations,  will  facilitate  the  application  of  new 
analytic  techniques.  The  emergence  of  cloud  computing  offers  researchers  the  potential  to  run 
larger  scale  data  analysis,  modeling,  and  simulations  that  were  once  impractical. 

4.3.  Analysis 

Extracting  information  from  unstructured  data  remains  a  major  research  challenge.  While  new 
methods  emerge  every  day,  researchers  need  efficient  and  effective  ways  to  represent  or  model 
data  arising  from  multiple,  heterogeneous  sources.  How  does  one  combine  video  analysis  with 
data  from  Twitter?  We  need  to  enhance  and  mature  the  mathematical  tools  that  make  true 
multisource  analysis  practical. 
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1.  Introduction 

Challenges  to  the  security,  health,  and  sustainable  growth  of  our  society  keep  escalating 
asymmetrically  due  to  the  accelerating  pace  of  global  change.  The  increasing  velocity  and  volume 
of  information  sharing,  social  networking,  economic  forces,  and  environmental  change  have 
expanded  the  number  and  frequency  of  "game-changing  moments"  that  a  community  can  face. 
Now  more  than  ever,  we  need  anticipatory  reasoning  technologies  based  on  sociocultural 
understanding  to  detect,  analyze,  and  forecast  potential  change  so  that  we  can  plan  appropriate 
interventions  to  neutralize  adversaries  and  protect  the  public  (Costa  &  Boiney,  2012).  The  creation 
of  such  a  "social  radar"  starts  with  the  detection  of  sociocultural  signatures  in  data  streams. 

By  harvesting  behavioral  data  and  analyzing  them  through  evidence-based  reasoning,  we  can 
detect  sociocultural  signatures  in  their  context  to  support  situation  awareness  and  decision 
making.  Developers  use  the  harvested  data  as  training  materials  from  which  to  infer  computational 
models  of  sociocultural  behaviors  or  calibrate  parameters  for  such  models.  Harvested  data  also 
serve  as  evidence  input  that  the  models  use  to  generate  insights  about  observed  and  future 
behaviors  for  targets  of  interest.  This  input  often  results  from  assembling  data  of  diverse  types  and 
aggregating  them  into  a  form  suitable  for  analysis. 

To  train  or  run  a  model,  we  must  analyze  data  needed  to  bring  out  the  categories  of  content 
relevant  to  the  domain  addressed.  If,  for  example,  we  are  using  messages  that  a  group  has 
broadcast  to  model  that  group's  intent  to  engage  in  violent  behavior,  we  must  process  those 
messages  to  extract  and  measure  indicators  of  violent  intent.  We  can  then  use  the  extracted 
indicators  and  the  associated  measurements  (e.g.,  rates  or  counts  of  occurrence)  to  train/calibrate 
and  run  computational  models  that  assess  the  propensity  for  violence  expressed  in  the  source 
message. 

Ubiquitous  access  to  the  Internet,  mobile  telephony,  and  technologies  such  as  digital  photography 
and  digital  video  have  led  to  the  emergence  of  social  media  application  platforms  such  as 
Facebook,  YouTube,  and  Twitter  that  are  altering  the  nature  of  human  social  interaction.  The  fast 
and  increasing  pace  of  online  social  interaction  introduces  new  challenges  and  opportunities  for 
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gathering  sociocultural  data.  Challenges  include  the  development  of  harvesting  and  processing 
techniques  tailored  to  these  new  data  environments  and  formats  (e.g.,  Twitter,  Facebook),  the 
integration  of  social  media  content  with  traditional  media  content,  and  the  protection  of  personal 
privacy.  As  modelers  address  these  and  other  challenges,  many  new  behavioral  data  and  data 
analysis  methods  become  available  that  are  shaping  social  computing  as  a  strongly  data-driven 
experimental  discipline  with  an  increasing  impact  on  the  decision-making  processes  of  groups  and 
individuals  alike. 

In  this  chapter,  we  review  current  advances  and  trends  in  the  detection  of  sociocultural  signatures. 
We  present  specific  embodiments  of  the  issues  discussed  with  respect  to  the  assessment  of  violent 
intent  and  sociopolitical  contention.  We  begin  by  reviewing  current  approaches  to  the  detection  of 
sociocultural  signatures  in  these  domains.  Next,  we  examine  novel  methods  for  harvesting  data 
from  social  media  content.  We  then  describe  the  application  of  sociocultural  models  to  social 
media  content,  and  conclude  by  commenting  on  current  challenges  and  future  developments. 

2.  Current  Approaches  to  Sociocultural  Signature  Detection 

This  section  defines  sociocultural  signatures  and  reviews  existing  methods  for  detecting  them.  For 
expository  purposes,  we  use  the  modeling  of  sociopolitical  contention  and  violent  intent  as  the 
case  study,  but  the  methods  described  apply  to  other  domains  as  well. 

We  distinguish  between  sociocultural  data  signatures  (SDSs)  and  sociocultural  model  signatures 
(SMSs).  An  SDS  can  be  envisioned  as  a  set  of  attribute-value  pairs  describing  a  data  record,  as 
defined  in  (1),  and  exemplified  in  (2)  with  reference  to  violent  intent  (vi),  where  attributes  denote 
classes  of  words  (e.g.,  military  =  { war ,  soldier ,  weapon...})  and  the  associated  values  indicate  how 
frequently  the  instances  of  the  attribute  occur  in  a  given  message. 

(1)  SDS\{DataRecord  =  #i,aj  =  vjf  ...,an  =  vn},n  >  i,j  >  0 

(2)  SDSvi:{TextID  =  6, military  =  54,  law  =  12,  religion  =  33, protest  =  25,  urge  = 

10,...} 

Attributes  represent  parameters  or  indicators  of  the  sociocultural  phenomenon  under  analysis.  The 
values  associated  with  these  attributes  either  quantify  the  presence/absence  of  the  attribute  or 
provide  a  qualitative  specification.  Quantitative  values  can  be  expressed  numerically  in  various 
ways,  including  counts  as  shown  in  (2),  percentages,  weights,  or  Boolean  values.  Qualitative  values 
provide  either  a  type  or  token  characterization  of  their  attributes.  Type  values  are  usually  defined 
as  categories  within  a  conceptual  scheme  such  as  an  ontology  (e.g.,  the  concept  "positive  mood") 
that  have  token  values  as  extensions  (e.g.,  "happy,  hopeful,  satisfied").  Token  values  are  the  real- 
world  entities:  for  example,  the  word  "happy,"  a  smiley  face,  the  sound  of  laughter,  or  a  video 
sequence  of  someone  smiling.  Values  can  be  assigned  to  attributes  manually  using  the  judgment  of 
subject  matter  experts,  or  automatically  using  data  mining  techniques  to  find  and  measure 
instances  of  attributes  in  datasets. 

An  SMS  is  a  function  that  takes  a  set  of  SDSs  as  input  and  then  detects  and  assesses  the  behavioral 
pattern  supported  by  the  SDS  set,  as  shown  in  (3)  and  exemplified  in  (4).  A  more  complex  example 
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of  an  SMS  is  the  decision  tree  shown  in  Figure  5,  which  uses  SDSs  extracted  from  messages  by 
violent  and  non-violent  groups  (e.g.,  those  shown  in  Figure  4)  as  training  materials  to  detect 
messages  from  terrorist  groups  and  their  characteristics. 

yields 

(3)  SMS :  {SDSJf=1 - >  Behavioral  Pattern  Assessment 

yields 

(4)  SMSvi:{ MILITARY  >=  36.5,violent_act  >3.5}  - >  p (terrorist_message)  = 

25.22% 


While  it  is  in  principle  possible  to  create  data  and  model  signatures  that  contain  all  social  and 
cultural  factors  known  to  be  relevant  to  human  behavior,  in  practice  the  size  and  generic  nature  of 
such  signatures  would  make  them  too  cumbersome  to  generate,  maintain,  and  apply.  Therefore, 
modelers  customarily  select  as  specific  a  focus  as  needed  to  address  the  domain  of  interest.  Even 
signatures  that  address  a  specific  behavioral  trait,  such  as  the  propensity  to  commit  acts  of 
violence,  exhibit  differences  in  behavioral  indicators  and  their  relative  import  according  to  the 
specific  domain  of  application,  e.g.,  workplace  violence  (White  &  Meloy,  2007),  terrorism 
(Monahan,  in  press;  Sanfilippo,  McGrath,  &  Bell,  2013;  Sanfilippo,  McGrath,  &  Whitney,  2011), 
mass  murder,  insider  threats  (Greitzer,  Kangas,  Noonan,  Dalton,  &  Hohimer,  2012),  spousal 
homicide,  and  public  figure  stalking/assassination  (Meloy,  Hoffman,  Guldimann,  &  James,  2012; 
Meloy,  Sheridan,  &  Hoffman,  2008;). 

2.1.  Acquisition  of  SDSs 

Content  analysis  is  perhaps  the  most  widely  used  technique  to  distill  SDSs  from  data  that  include 
surveys,  interviews,  ethnographies,  social  media,  news  wires,  and  public  speeches.  This 
methodology  interprets  text  through  categorical  annotation  to  study  the  content  of 
communication  (Holsti,  1969;  Krippendorff,  2004).  For  example,  assessing  violent  intent  through 
content  analysis  involves  identifying  categories  of  meaning  correlated  with  the  expression  of 
violent  behavior  or  the  lack  thereof.  Analysts  then  draw  inferences  from  the  occurrence  of  such 
categories  in  the  document(s)  reviewed  to  estimate  the  likelihood  that  the  communication  source 
would  engage  in  violent  behavior. 

Traditional  content  analysis,  where  content  categories  are  defined  as  sets  of  words  based  on 
explicit  rules  of  coding,  relies  on  manual  annotation.  Smith,  Suefeld,  Conway,  &  Winter  (2008), 
Winter  (2011),  Suedfeld  &  Brcic  (2011),  and  Conway,  Gornick,  Houck,  Hands  Towgood,  &  Conway 
(2011)  provide  examples  of  content  analysis  studies  based  on  manual  annotation.  Others  (Borum, 
Bartel,  &  Forth,  2006)  use  structured  professional  judgment  to  generate  assessments  based  on 
subject  matter  experts'  answers  to  questionnaires.  For  example,  the  Violent  Extremist  Risk 
Assessment  instrument  (VERA;  Pressman,  2009)  determines  the  risk  that  an  individual  would 
engage  in  terrorism,  either  as  part  of  a  group  or  as  a  "lone  wolf,"  in  terms  of  ideological,  religious, 
and  political  motivations,  using  guided  expert  judgment  based  on  responses  to  a  previously  set  list 
of  attributes,  as  shown  in  Figure  1. 
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Figure  1.  VERA  Coding  Response  Form.  Adapted  with  permission  from  Pressman  (2009). 


The  increased  ubiquity  and  reliability  of  text  mining  techniques  has  led  to  progressive  automation 
of  the  annotation  task.  Text  mining  is  the  process  of  discovering  information  in  large  text 
collections  and  automatically  identifying  interesting  patterns  and  relationships  in  textual  data.  This 
relatively  new  and  highly  interdisciplinary  research  area,  which  has  recently  caught  the  interest  of 
many  in  the  research  and  industry  communities  (mainly  due  to  the  ever-increasing  amount  of 
information  available  on  the  web  and  elsewhere),  brings  together  research  insights  from  the  fields 
of  data  mining,  natural  language  processing,  machine  learning,  and  information  retrieval.  Text 
mining  is  related  to  data  mining— an  older  research  area  focused  on  extracting  significant 
information  from  data  records— but  has  proven  more  difficult  because  the  source  data  consist  of 
unstructured  collections  of  documents  rather  than  structured  databases.  Many  applications  now 
utilize  text  mining,  including  question-answering  applications,  automatic  construction  of  databases 
on  job  postings,  and  dictionary  construction.  Feldman  &  Sanger  (2007)  provide  a  thorough  survey 
of  research  in  the  area  of  text  mining  and  the  ensuing  language  analysis  capabilities. 

Linguistic  analyses  of  affective  content  facilitated  by  text  mining  may  reveal  emotion,  mood, 
behavior,  attitude,  cognitive  state,  physical  state,  hedonic  signal,  and  sensations.  In  addition, 
cultural  artifacts  extracted  from  such  media  may  point  to  useful  demographic  indicators  that  have 
potential  utility  for  the  detection  of  sociocultural  and  psychosocial  signatures  (Ortony,  2003).  Such 
signatures  can  vary  considerably  in  complexity.  For  example,  Pennebaker  (2011)  uses  the 
occurrence  of  function  words  to  characterize  texts  authored  by  terrorist  groups  as  compared  to 
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non-terrorist  radical  groups.  Texts  from  terrorist  communication  sources  contain  a  statistically 
significant  higher  number  of  personal  pronouns,  while  texts  from  non-terrorist  communication 
sources  contain  a  statistically  significant  higher  number  of  articles.  Function  words  can  also  be 
grouped  into  higher  order  variables,  such  as  the  variable  status,  which  measures  the  presence  of 
"we-words  and  you-words"  and  absence  of  "l-words." 

Other  methods  for  extracting  sociocultural  and  psychosocial  signatures  using  text  mining 
techniques  are  based  on  specific  theoretical  approaches  to  social  and  political  analysis,  such  as 
Leadership  Traits  Analysis  (LTA;  Hermann,  2003),  operational  code  analysis  (Walker,  1990),  and 
social  movement  theory  (McAdam,  McCarthy,  &  Zald,  1996;  McAdam,  Tarrow,  &  Tilly,  2001).  Trait 
leadership  is  defined  as  integrated  patterns  of  individual  behavioral  characteristics  that  identify  the 
effectiveness  of  leadership  styles  across  a  variety  of  group  and  organizational  situations  (Hermann, 
2003;  Zaccaro,  Kemp,  &  Bader,  2004).  Hermann  and  Sakiev  (2011)  evaluate  leadership  styles  along 
seven  dimensions— control,  power,  conceptual  complexity,  self-confidence,  task  orientation, 
distrust  of  others,  and  in-group  bias  -  to  detect  terrorist  rhetoric.  They  identified  these  seven  traits 
in  public  statements  from  political  leaders  and  measured  them  using  the  LTA  program  embedded 
in  the  Profiler  Plus  software  (Social  Science  Automation,  2013;  Young,  2001). 

Operational  code  refers  to  the  values,  world  views,  and  response  repertoire  that  an  individual 
acquires  and  shares  with  other  members  of  an  organization.  Walker  (2011)  examines  the  violent 
propensities  of  groups  and  leaders  in  terms  of  their  overall  political  philosophies  and  their  views  on 
how  best  to  achieve  them.  This  examination  is  based  on  automated  content  analysis  that  used  the 
Verbs  In  Context  System  (VICS)  to  retrieve  and  analyze  the  operational  codes  of  leaders  and  groups 
from  their  public  statements.  As  implemented  in  the  Profiler  Plus  software  (Young,  2001),  VICS 
identifies  the  use  of  various  forms  of  power  attributed  to  the  self  and  others  in  a  speaker's 
rhetoric.  The  system  conceptualizes  the  exercise  of  power  in  terms  of  classes  of  events  (i.e., 
reward,  promise,  appeal/support,  oppose/resist,  threaten,  punish).  Walker's  findings  indicate  that 
terrorists  have  a  deterministic  view  of  the  political  universe;  they  also  believe  they  exert  greater 
historical  control  over  world  events,  are  less  accepting  of  risk,  and  are  more  prone  to  choose 
conflict  over  cooperation  than  non-terrorist  groups. 

Social  movement  theory  (McAdam  et  al.,  1996,  2001)  explains  the  emergence  of  sociopolitical 
violence  in  terms  of  mobilizing  structures,  political  opportunities,  and  cultural  framing  (della  Porta, 
2008;  McAdam  et  al.,  1996;  Wiktorowicz,  2003;).  The  analysis  of  cultural  framing  unveils  the 
communicative  and  mental  processes  that  explain  how  social  movement  entrepreneurs  endeavor 
to  influence  their  target  audiences,  and  how  the  target  audiences  respond.  Sanfilippo  et  al.  (2011) 
combine  insights  from  cultural  framing  and  theories  that  explain  the  emergence  of  violence  in 
terms  of  moral  disengagement,  the  violation  of  sacred  values,  and  social  isolation  in  order  to  detect 
violent  intent.  For  instance,  communicative  strategies  that  involve  feelings  of  moral  disengagement 
such  as  hate  and  disgust  and  focus  on  religion  and  military  topics  tend  to  be  more  highly  correlated 
with  texts  from  terrorist  sources. 
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2.2.  SMS  Development  and  Evaluation 

Modelers  have  used  diverse  techniques  to  develop  SMSs.  Some  models  are  primarily  developed 
manually,  using  insights  from  social  science  theories.  Developers  can  instantiate  these  models  with 
evidence  from  SDSs  to  provide  operational  assessments  as  outlined  in  (3)  and  (4).  This  evidence  can 
be  gathered  either  manually  (e.g.,  through  subject  matter  expert  judgments)  or  automatically 
through  data  mining  and  content  analysis  (see  section  2.1).  Other  models  are  derived  from  SDSs 
using  machine  learning  methods;  as  discussed  in  section  2.1.,  these  signatures  are  distilled  from 
raw  data  through  content  analysis  processes  that  embody  insights  from  social  science  theories. 

2.2.1.  Manually  developed  models 

Chaturvedi  et  al.  (2005)  present  an  agent-based  model  of  insurgency  in  Indonesia  based  on  the 
analysis  and  simulation  of  dynamic  interrelationships  among  grievances,  level  of  resources,  and 
organizational  capacity  to  mobilize  members  toward  insurgency.  The  model  draws  on  insights  from 
resource  mobilization  theory  (McAdam,  Tarrow,  &  Tilly,  2001;  McCarthy  &  Zald,  2001);  see  Figure 
2.  Equations  that  regulate  inputs  and  outcomes  express  dynamic  relationships  among  relevant 
factors.  For  example,  equations  define  an  agent's  intention  to  rebel  (I)  as  a  function  of  grievance 
(G)  and  risk  propensity  (RP):  /  =  f(G,  RP ).  This  approach  requires  the  modelers  to  calibrate  agents 
with  evidence  relevant  to  resource  mobilization  theory  (e.g.,  resources,  organization,  political 
opportunities,  discontent,  and  contention).  This  evidence  is  mined  from  open  source  repositories 
such  as  Polity  IV,  the  Indonesia  Public  Opinion  Survey  2005,  the  CIA  World  Fact  Book, 
Worldpress.org,  and  Europa  Magazine. 

One  of  the  advantages  of  this  modeling  approach  is  that  it  overcomes  the  difficulty  of 
understanding  situations  (e.g.,  insurgency  in  Indonesia)  for  which  few  comparable  and 
generalizable  historical  cases  exist.  For  example,  researchers  might  detect  insurgency  patterns  in 
Indonesia  through  simulation  in  a  virtual  world,  where  agents  interact  according  to  behavioral 
patterns  derived  from  social  science  theories  (e.g.,  resource  mobilization  theory)  and  informed  by 
evidence  mined  from  open  source  data. 
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Figure  2.  Agent-based  model  of  insurgency.  Adapted  with  permission  from  Chaturvedi  et  al.  (2005). 


Sticha,  Buede,  &  Rees  (2005)  describe  an  application  of  Bayesian  nets  that  enables  analysts  to 
perform  anticipatory  reasoning  about  a  subject's  decision-making  process  by  using  personality 
factors  derived  from  LTA  (Hermann,  2003)  and  the  Neuroticism-Extroversion-Openness  (NEO) 
Inventory  (Costa  &  McCrae,  1985)  as  indicators  (Figure  3).  The  relationships  among  indicators  are 
expressed  as  probabilities  that  indicate  whether  and  the  extent  to  which  an  indicator  may  cause 
value  changes  in  other  indicators.  The  application  calibrates  these  relationships  by  aggregating 
subject  matter  judgments.  Further,  it  instantiates  indicators  with  evidence  drawn  from  (a)  running 
the  NEO  test  on  input  forms  compiled  by  subject  matter  experts,  and  (b)  processing  first-person 
verbalizations  (e.g.,  speeches  and  interviews)  with  Profiler  (Young,  2001)  to  obtain  leadership 
personality  profiles. 
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Figure  3.  Bayesian  net  model  of  a  leader's  decision-making  process.  Adapted  with  permission  from 
Sticha  et  al.  (2005). 


The  agent-based  model  discussed  by  Sticha  et  al.  (2005)  is  developed  through  expert  judgment  and 
guided  by  insights  from  social  science  theory.  However,  in  this  case  the  modelers  use  specific 
processes  to  extract  evidence  from  relevant  data  sources  (e.g.,  NEO  test  and  Profiler+).  This 
approach  is  particularly  useful  when  the  relationships  among  indicators  cannot  be  deterministically 
established  and  are  thus  best  expressed  probabilistically,  and  when  specific  evidence  mining 
techniques  can  be  applied  to  inform  model  indicators. 

2.2.2.  Models  derived  from  SDSs 

In  the  modeling  approaches  described  in  section  2.2.1.,  developers  create  models  manually  and 
then  use  evidence  from  SDSs  as  input  to  verify  how  the  models  respond  to  specific  circumstances, 
e.g.,  use  recent  communications  released  by  the  group  to  detect  the  group's  propensity  to  engage 
in  violent  behavior.  Developers  can  also  use  those  SDSs  to  derive  the  model  automatically  from 
available  historical  data  through  machine  learning  methods.  In  this  case,  the  SDSs  embody  the 
insight  from  social  science  theory  and  subject  matter  expertise,  and  the  model  infers  the 
relationships  between  factors  and  their  outcomes  algorithmically  from  the  signatures. 
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For  example,  Sanfilippo  et  al.  (2011)  describe  how  the  propensity  and  timeframe  for  engaging  in 
violent  behavior  can  be  modeled  using  decision  trees  to  analyze  communication  patterns.  They 
extracted  SDSs  that  describe  communication  strategies  from  messages  authored  by  targets  of 
interests  using  an  approach  that  combines  content  analysis  and  text  mining.  The  underlying 
category  scheme  blends  insights  from  frame  analysis  (Benford  &  Snow,  2000;  Entman,  2004; 
Gamson,  1992;  Goffman,  1974),  speech  act  theory  (Austin,  1962;  Searle,  1969),  and  theories  that 
explain  the  emergence  of  violence  with  reference  to  moral  disengagement  (Bandura,  1999),  the 
violation  of  sacred  values  (Tetlock,  Kristel,  Elson,  Green,  &  Lerner,  2000),  and  social  isolation 
(Navarro,  2009).  This  category  scheme  is  enriched  with  a  lexicon,  as  exemplified  in  Table  1,  to 
enable  automated  document  annotation  using  text  mining  methods  (Sanfilippo,  2007).  This  yields 
SDSs  where  each  analyzed  record  (e.g.,  a  document)  is  encoded  as  an  array  of  values  for  the 
selected  behavioral  indicators,  as  shown  in  Figure  4.  Model  developers  can  assess  the  validity  of 
the  category  scheme  as  a  data  annotation  device  through  experiments  on  inter-annotator 
agreement  in  which  a  set  of  human  annotators  use  the  category  scheme  to  mark  the  same  data. 
The  resulting  markings  are  then  compared  to  measure  the  level  of  agreement,  using  measures  such 
as  the  kappa  score  (Sanfilippo  et  al.,  2007). 

Using  supervised  machine  learning  techniques  with  SDSs  extracted  from  training  datasets  (Figure  4) 
as  training  data,  developers  then  build  classification  models  capable  of  detecting  communications 
that  signal  the  propensity  for  violence  and  providing  a  categorical  breakdown  of  such  an 
assignment.  For  example,  the  alternating  decision  tree  model  shown  in  Figure  5  provides  a 
detection  model  for  terrorist  and  radical  non-terrorist  messages  in  terms  of  frame-based  SDSs 
(Table  1  and  Figure  4). 
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Figure  4.  SDSs  derived  from  a  collection  of  documents  (e.g.,  speeches,  interviews,  book  chapters, 
and  newswires).  Each  row  below  the  headers  represents  a  document;  each  column  represents  a 
feature  or  category  used  for  sociocultural  analysis  and  the  values  of  that  feature  in  each  document. 
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Table  1.  Partial  representation  of  the  Frames  in  Action  annotation  scheme 


Top  Level  Categories 

Intermediate  Categories 

Lexical  Extensions 

Message  Delivery 

ILLUSTRATE 

help ,  aid  assist, ... 

TRANSFER  MESSAGES 

tell,  explain,  cite, ... 

INSTRUMENTAL  COMMUNICATION 

call,  sign,  broadcast, ... 

Quest  for  Resonance 

HELP 

help,  aid  assist, ... 

DEFEND 

protect,  defend, ... 

EQUIP 

reward,  arm, ... 

APPROVE 

accept,  encourage 

PAY 

pay,  serve, ... 

GIVE 

give,  render, ... 

ADMIRE 

support,  love,  trust, ... 

FREE 

liberate,  absolve, ... 

Call  to  Arms 

URGE 

pledge,  obligate,  bind, ... 

Violence  &  Contention 

DISAPPEAR 

die,  vanish, ... 

PAIN 

hurt,  bother, ... 

POCKET 

imprison,  trap, ... 

VIOLENT  ACT 

frighten,  attack,  destroy, ... 

STOP 

end,  vanish,  kill, ... 

PROTEST 

appeal,  reject,  argue, ... 

Social  Isolation 

SPATIAL  CONFIGURATION 

stand,  lie,  sit, ... 

LODGE 

stay,  reside,  dwell 

WITHDRAW 

retreat,  retire, ... 

CONFINE 

commit,  confine, ... 

CONCEAL 

hide,  isolate, ... 

Moral  Disengagement 

MANNER  OF  SPEAKING 

cry,  scream,  shout, ... 

JUDGE 

attack,  punish, ... 

CRITICIZE 

denounce,  condemn, ... 

MARVEL 

suffer,  fear,  wonder 

Violation  of  Sacred  Values 

MILITARY 

war,  army,  soldier, ... 

RELIGION 

god,  church,  mosque, ... 

Note :  Adapted  from  Sanfilippo  et  al.  (2011) 


In  decision-tree  classification  (Quinlan,  1986),  the  model  identifies  members  of  a  class  as  the  result 
of  a  sequence  of  decisions.  As  shown  in  Figure  5,  a  decision  tree  typically  consists  of  two  types  of 
nodes:  test  nodes  and  prediction  nodes.  The  test  node  describes  the  condition  that  must  be  met  in 
order  to  make  a  decision— e.g.,  military  <  36.5— and  the  prediction  node  specifies  the  results  of  the 
decision  made— e.g.,  the  numerical  contribution  of  the  test  node  "military  <  36.5"  to  the 
recognition  of  the  non-terrorist  class  (2.25%).  Several  test  nodes  can  occur  in  a  sequence  to 
indicate  the  number  of  decisions  that  must  be  taken  and  the  order  in  which  these  decisions  follow 
one  another  to  reach  a  prediction  outcome,  e.g., 


[MILITARY  >=  36.5]  ->  [VIOLENT_ACT  >  3.5]  ->  p {terrorist  message)  =  25.22% 
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Prediction  Nodes 

i  est  wooes 

Non-Terrorist  Message 

Terrorist  Message 

|  (l)MILITARY  <  36.5 

2.25% 

|  |  (7)EVENT REMOVE  <  0.5 

3.29% 

|  |  |  (10)EVENT MANNER SPEAKING  <  0.5 

2.19% 

|  |  |  (10)EVENT MANNER SPEAKING  >=  0.5 

12.06% 

|  |  (7)EVENT REMOVE  >=  0.5 

12.82% 

|  (l)MILITARY  >=  36.5 

19.81% 

|  |  (2)EVENT VIOLENT ACT  <  3.5 

17.33% 

|  |  (2)EVENT VIOLENT ACT  >=  3.5 

25.22% 

|  (3)ADMINISTRATION  <  8.5 

3.72% 

|  |  (8)MILITARY  <  0.5 

14.37% 

|  |  (8)MI LITARY  >=  0.5 

2.72% 

|  (3)ADMINISTRATION  >=  8.5 

7.32% 

|  |  (6)EVENT ILLUSTRATE  <  0.5 

7.36% 

|  |  (6)EVENT ILLUSTRATE  >=  0.5 

15.66% 

|  (4)MILITARY<  10.5 

5.33% 

|  |  (5)POLITICS<  11.5 

3.70% 

|  |  |  (9)EVENT BELIEVE  <  0.5 

7.65% 

|  |  |  (9)EVENT BELIEVE  >=  0.5 

8.09% 

|  |  (5)POLITICS  >=  11.5 

22.92% 

|  (4)MI LITARY  >=  10.5 

6.17% 

Figure  5.  Alternating  decision  tree  model  that  can  be  used  to  detect  communications  from  terrorist 
and  non-terrorist  radical  groups.  A  prediction  value  indicates  the  importance  of  each  decision  step. 
For  ease  of  interpretation,  prediction  values  have  been  normalized  as  percentages. 


To  learn  a  decision  tree  classifier  from  a  training  dataset  such  as  the  one  exemplified  by  Figure  4,  a 
model  establishes  the  sequential  order  of  test  nodes  according  to  how  informative  the  nodes' 
attributes  are.  The  model  determines  the  information  content  of  an  attribute  by  its  information 
gain  with  respect  to  the  classification  tasks  (Quinlan,  1986;  Mitchell,  1997).  The  information  gain  of 
an  attribute  with  respect  to  a  class  is  the  reduction  in  entropy  (i.e.,  uncertainty)  of  the  value  for  the 
class  when  we  know  the  value  of  the  attribute.  The  test  nodes  with  more  informative  attributes 
occur  earlier  in  the  decision  tree.  The  model  creates  test  nodes  using  the  available  attributes  until 
all  data  in  the  training  dataset  have  been  accounted  for.  Typically,  not  all  attributes  are  used 
because  decision  tree  learners  use  pruning  strategies  to  reduce  the  number  of  nodes.  The  number 
of  attributes  depends  on  the  specific  implementation.  The  alternating  decision  tree  algorithm 
(Freund  &  Mason,  1999;  Flolmes,  2001)  uses  a  machine  learning  meta-algorithm  called  boosting  to 
minimize  the  number  of  nodes  without  losing  accuracy. 

Researchers  can  use  the  decision  tree  model  shown  in  Figure  4  to  detect  messages  from  terrorist 
sources  and  identify  communicative  frames  in  terrorist  messages.  Messages  are  first  mapped  into 
SDSs,  such  as  the  one  shown  in  Figure  4,  using  automated  content  analysis.  The  model  then 
processes  the  emerging  data  signatures,  which  describe  messaging  strategies,  to  establish  which 
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test  nodes  match  the  data  signatures  and  to  generate  a  detection/prediction  (e.g.,  terrorist  or  non¬ 
terrorist  message). 

Knowledge  of  the  messaging  strategies  that  terrorists  use  to  influence  their  target  audiences  helps 
operational  users  to  design  strategies  that  neutralize  the  communicative  reach  of  the  terrorists.  For 
example,  realization  that  al  Qa'ida  (AQ)  has  suddenly  begun  to  focus  its  propaganda  on  young 
Muslims  in  Pakistan  using  frames  of  "sacred  values  violation"  would  highlight  relevant 
geodemographic  targets  and  topics  that  U.S.  planners  should  address  to  design  a  strategy  that 
neutralizes  AQ's  communicative  reach. 

No  matter  how  ideologically  and  operationally  close  two  groups  may  be,  they  voice  their 
communication  strategies  differently  to  suit  the  mindset  of  the  social  and  cultural  audience  they 
wish  to  reach.  For  example,  the  transnational  Islamist  militant  organization  AQ  and  its  subordinate 
group  AQ  in  the  Arabian  Peninsula  (AQAP)  pursue  a  terrorist  agenda  based  on  the  same 
fundamentalist  doctrine.  However,  while  AQAP  directs  its  radicalization  efforts  to  a  specific  cultural 
milieu  (the  Arabian  Peninsula),  AQ's  radicalization  efforts  have  a  more  overarching  goal.  The  model 
shown  in  Figure  5  captures  shared  motivations  and  intents  of  AQ  and  AQAP  as  compared  to  radical 
non-terrorist  Islamist  organizations,  but  does  not  tell  us  anything  specific  about  AQ  or  AQAP.  U.S. 
planners  therefore  need  a  comparative  analysis  of  the  communicative  strategies  used  by  AQ  and 
AQAP  to  design  counterterrorist  influence  strategies  informed  by  cultural  differences  between  the 
two  groups. 

Can  the  sociocultural  data  and  model  signature  framework  used  to  develop  the  model  shown  in 
Figure  4  be  adjusted  to  address  differences  between  groups  that  share  the  same  values  and 
attitudes  toward  employing  violence  but  operate  in  different  cultural  realities?  Sanfilippo  & 
McGrath  (2011)  show  that  the  same  annotation  scheme  used  for  the  violent  intent  detection 
model  described  in  Sanfilippo  et  al.  (2011)  (analogous  to  the  one  shown  in  Figure  5)  can  be  used  to 
construct  detection  models  that  focus  on  how  individual  terrorist  groups  communicate  with  their 
target  audiences.  As  illustrated  in  Figure  6,  the  detection  task  in  this  case  involves  recognizing 
whether  a  specific  communication  effort  originates  from  AQ  or  AQAP,  identifying  the  framing 
features  that  contribute  to  one  outcome  or  the  other,  and  determining  the  extent  to  which  they  do 
so.  For  example,  the  model  reveals  a  strong  correlation  between  group-member  authorship  and 
AQAP  messages,  and  between  group-leader  authorship  and  AQ  messages.  Also,  AQAP 
radicalization  propaganda  is  more  likely  to  target  young  Muslims,  whereas  AQ  focuses  on  a  wider 
set  of  audiences  (e.g.,  the  Muslim  faithful  as  a  whole  or  some  regional  subset,  as  well  as  the  United 
States  and  its  allies).  AQAP  messages  emphasize  fulfillment,  while  AQ's  messages  focus  on  rejection 
and  assertion. 

We  can  now  combine  insights  about  the  framing  processes  germane  to  each  terrorist  group  with 
those  from  the  earlier  models  that  tell  us  about  communicative  strategies  derived  from  common 
motivations  and  intents  of  terrorists.  The  ensuing  methodology  creates  a  sociocultural  analytic 
framework  that  captures  both  general  and  culture-specific  aspects  of  terrorist  rhetoric,  and 
supports  the  design  of  strategic  influence  interventions  for  terrorism  deterrence. 
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|  (6)Event REJECT  <  0.5 

4.51% 

|  (6)Event REJECT  >=  0.5 

9.06% 

|  (7)author  =  member 

13.27% 

|  (7)author  !=  member 

3.90% 

Figure  6.  Alternating  decision  tree  model  (Freund  &  Mason,  1999)  that  can  be  used  to  detect 
communications  from  AQAP  and  AQ  on  the  basis  of  frame  annotations.  A  prediction  value  indicates 
the  importance  of  each  decision  step.  For  ease  of  interpretation,  prediction  values  have  been 
normalized  as  percentages. 


When  sufficient  and  reliable  historical  data  are  available,  the  automated  inference  of  SMSs  from 
data  can  help  complement  and  enhance  human  intelligence.  Qualities  such  as  the  ability  to  focus 
on  what  is  perceived  to  be  most  important  and  the  capacity  to  make  quick  decisions  on  the  basis  of 
insight  and  intuition  make  human  judgment  uniquely  effective  (Gigerenzer,  2007;  Gladwell,  2005) 
and  have  great  value  in  analysis  and  detection.  However,  the  same  qualities  can  also  lead  to 
fallacious  reasoning  when  judgment  is  affected  by  memory  limitations  (Miller,  1956),  lack  of 
knowledge/expertise  (Klein,  1998),  biases  due  to  factors  such  as  increased  confidence  in  extreme 
judgments  and  highly  correlated  observables  (Kahneman  &  Tversky,  1973),  positive  framing 
(Tversky  &  Kahneman,  1981),  "groupthink"  (Jani,  1972;  Surowiecki,  2004),  and  premature 
commitment  to  a  single  expected  outcome  (Heuer,  1991).  Machine  learning  methods  can  help  to 
counterbalance  these  weaknesses  in  human  judgment  by  providing  an  unbiased  framework  which 
can  then  be  steered  by  human  intelligence. 
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Decision  trees  offer  a  rather  explicit  and  intuitive  classification  device  for  interpreting  models 
learned  from  the  data,  as  opposed  to  other  algorithms  such  as  support  vector  machines  (Vapnik, 
1995)  that  produce  equal  or  sometimes  rivaling  results,  but  whose  output  is  more  difficult  to 
interpret.  Depending  on  the  nature  of  the  data,  other  machine  learning  algorithms  may  offer  better 
solutions,  as  discussed  in  the  next  section. 

2.3.  SMS  Evaluation 

Regardless  of  which  technique  modelers  adopt  for  the  detection  of  sociocultural  signatures,  they 
must  evaluate  the  resulting  models  to  obtain  indications  of  reliability  in  the  sense  of  accuracy. 
Model  accuracy  is  typically  measured  by  comparing  the  model's  expectations  against  historical 
data.  To  increase  the  significance  of  the  tests,  the  comparison  uses  a  portion  of  the  training  data 
reserved  for  testing.  Precision,  recall,  and  the  F-measure— defined  in  terms  of  true 
positives/negatives  and  false  positives/negatives— provide  examples  of  the  methods  used  to 
evaluate  model  accuracy.  As  shown  in  Figure  7,  the  terms  true  positive/negative  and  false 
positive/negative  describe  whether  the  model's  class  assignments  for  the  test  data  match  the 
classes  observed  in  the  corresponding  ground  truth  data.  The  terms  positive  and  negative  refer  to 
the  class  predicted/expected  by  the  model  (e.g.,  ±  terrorist),  and  the  terms  true  and  false  establish 
whether  the  class  predicted  by  the  model  matches  the  actual  observed  class. 


Class  observed  in  the  historical  data  (ground  truth) 

Class  predicted  or  expected 
by  the  model 

TP  (true  positive) 

Correct  result 

FP  (false  positive) 

Incorrect  presence  of  result 

FN  (false  negative) 

Missing  result 

TN  (true  negative) 

Correct  absence  of  result 

Figure  7.  True/false  positives  and  negatives. 


As  shown  in  (5)  and  (6),  precision  decreases  as  the  rate  of  false  positives  (e.g.,  false  alarms) 
increases,  and  recall  decreases  as  the  rate  of  false  negatives  (e.g.,  misses)  increases.  The  F-measure 
provides  a  weighted  average  of  precision  and  recall,  as  shown  in  (7). 


(5]  precision  = 


TPs 

TPs+FPs 


(6) 

(7) 


recall  = 


TPs 

TPs+FNs 


F-measure  = 


2  * 


precision*recall 

precision+recall 


Evaluation  helps  users  to  understand  a  model's  reliability.  For  example,  the  evaluation  results  of 
the  alternating  decision  tree  model  in  Figure  6  indicate  that  communications  from  AQAP  and  AQ 
can  be  distinguished  with  high  accuracy  (Figure  8),  both  in  terms  of  precision  (low  occurrence  of 
false  positives)  and  recall  (low  occurrence  of  false  negatives). 
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AQC  AQAP 


■  Precision 
□  Recall 

■  F-Measure 


Figure  8.  Assessing  and  comparing  model  performance  using  precision  and  recall  measures. 


Evaluation  also  aids  users  to  understand  the  relative  reliability  of  models  derived  from  the  same 
data  but  using  different  algorithms.  For  example,  a  comparative  evaluation  of  the  alternating 
decision  tree  (ADT)  classifier  shown  in  Figure  5  with  a  multinomial  Bayesian  net  (MBN)  classifier 
(McCallum  &  Nigam,  1998)  built  by  training  on  the  same  data  (Sanfilippo  et  al.,  2013)  shows  that 
the  MBN  provides  higher  accuracy  (Figure  9).  Similar  to  the  tree  decision  model  discussed  earlier 
(Figure  5),  the  input  to  a  multinomial  Bayesian  classifier  is  a  document  represented  as  a  set  of 
attribute-value  pairs  indicating  the  number  of  times  each  feature  occurs  in  a  document  (see  Figure 
4).  To  classify  a  document,  the  MBN  algorithm  selects  the  class  that  has  the  highest  probability 
given  the  feature  makeup  of  the  document,  as  determined  by  the  document-class  correlations 
observed  in  the  training  dataset.  MBN  classifiers  are  particularly  effective  when  dealing  with 
textual  data.  They  generate  output  that  users  can  easily  interpret  in  terms  of  the  probability  that  a 
term  contributes  to  one  class  rather  than  the  other,  as  illustrated  in  Figure  10  with  reference  to  the 
violent  intent  model  under  discussion. 


Figure  9.  Assessing  and  comparing  model  performance  using  precision  and  recall  measures. 
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Figure  10.  Probabilities  relative  to  the  terrorist  and  non-terrorist  classes. 


While  it  is  always  preferable  to  use  sizeable  datasets  to  train  and  test  models  (e.g.,  datasets  with 
thousands  of  samples  for  each  class  to  be  detected),  this  is  not  always  possible,  especially  when 
manual  annotation  is  involved.  When  only  a  few  hundred  samples  are  available  for  each  class  to  be 
detected,  as  in  the  ADT  and  MBN  models  discussed  in  this  section,  modelers  often  use  ten-fold 
cross  validation  to  maximize  the  size  of  the  training  dataset.  In  /i-fold  cross-validation,  a  randomly 
selected  n%  of  the  training  data  is  reserved  for  testing  purposes  n  times;  the  model  then 
aggregates  the  resulting  n  evaluation  results  into  a  single  result  using  the  average  function.  This 
enables  model  developers  to  use  nearly  the  full  dataset  for  training,  but  still  test  the  model  on  a 
significant  number  of  data  examples  not  used  for  training. 

3.  Data  Harvesting:  Leveraging  Social  Media 

The  recent  expansion  of  social  media  data  types  and  sources  has  produced  a  large  variety  of  social 
media  content  that  researchers  can  use  in  addition  to  more  traditional  types  of  data  for 
sociocultural  modeling.  But  before  detection,  forecasting,  or  other  types  of  analysis  can  begin, 
researchers  must  obtain,  normalize,  and  manage  various  forms  of  data,  including  social  media  data. 

Social  media  content  serves  as  a  source  of  real-time,  spontaneous  data  on  a  wide  variety  of  issues 
(Sakaki,  Okazaki,  &  Matsuo,  2010).  The  content  varies  by  type  (multimedia  vs.  text)  and  by  nature 
(links  in  a  social  network  vs.  a  long  weblog  posting  vs.  a  short  tweet  of  a  Uniform  Resource  Locator 
[URL])  (Russel,  2011).  The  data  themselves  are  diverse  and  unrefined,  and  are  generated  in  massive 
quantities  (tens  of  thousands  of  records  per  second)  and  in  real  time.  This  real-time  quality 
provides  a  significant  advantage  for  situation  awareness  when  used  appropriately.  The  unrefined 
nature  of  the  data  offers  a  fresh  perspective  on  issues  also  addressed  in  official  sources.  Many 
forms  of  social  media  data  are  produced  in  streaming  environments,  but  researchers  must  evaluate 
them  in  tandem  with  batches  of  traditional,  refined  data  sources,  and  must  do  so  within  the 
constraints  of  personal  privacy  protection  guidelines  that  govern  the  use  and  merging  of  these  data 
and  sources. 

Most  public  means  of  accessing  the  data  place  limits  on  the  rate  of  data  that  can  be  harvested,  on 
the  fields  present  in  the  data  harvested,  and  on  the  distribution  and  use  of  the  data.  Social  media 
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content  appears  on  a  large  variety  of  platforms,  each  of  which  comes  with  its  own  constraints  and 
policies  about  obtaining,  using,  and  distributing  data.  While  commercial  means  of  obtaining  the 
data  overcome  some  of  these  issues,  they  impose  significant  costs  and  technical  constraints  on  the 
harvesting  pipelines.  Many  of  these  commercial  means  are  subject  to  the  same  constraints  as  the 
general  public  in  terms  of  obtaining  data  from  social  media  providers.  Some  of  these  commercial 
companies  add  value  by  providing  a  single  source  for  obtaining  data  (across  multiple  sources)  and 
translating  the  data  into  a  common  format  for  reasoning. 

As  noted,  social  media  data  typically  vary  in  format,  so  modelers  must  normalize  the  data  into  a 
universal  schema  for  further  analysis.  This  schema  must  include  elements  shared  by  other  types  of 
data  to  enable  analysis  across  data  types.  Normalization  is  non-trivial,  as  differences  in  language, 
differences  in  metadata  and  data  formats  (XML  [extensible  Markup  Language]  vs.  JSON  [JavaScript 
Object  Notation]  vs.  HTML  [Hypertext  Markup  Language]),  and  the  time  sensitivity  of  some  of  the 
embedded  data  (e.g.,  links  and  shortened  URLs)  require  a  robust  processing  pipeline  for 
transforming,  cleaning,  normalizing,  and  validating  the  harvests.  The  robustness  of  any  harvesting 
tool  is  critical  in  terms  of  obtaining  complete  data.  Social  media  content  is  often  distributed  at  the 
moment  it  is  generated.  With  some  systems  and  providers,  retrieving  data  from  a  live  stream  after 
it  has  been  missed  (e.g.,  because  of  server  downtime,  an  anomaly  in  the  harvesting  software,  etc.) 
is  burdensome  at  best.  Other  providers  simply  cannot  obtain  such  data.  Yet  the  lack  of  these  data 
can  alter  and  bias  results  in  each  subsequent  stage  of  sociocultural  behavior  sensemaking. 

A  normalization  stage  typically  follows  the  harvesting  of  raw  social  media  content.  Because  some 
providers  format  part  of  the  data  by  breaking  individual  pieces  of  data  into  small  elements  while 
others  present  the  data  as  a  single,  complex  element  with  a  different  representation,  significant 
challenges  may  arise  in  merging  various  content  types  into  a  single  representation  that  will  work 
for  all  of  the  desired  analysis.  Data  containing  elements  not  present  in  every  record  (e.g., 
geolocation)  present  another  technical  challenge.  Any  analysis  on  this  type  of  data  must  either 
supply  the  incomplete  information  through  approximation/reasoning  or  must  ignore  this  aspect. 

Once  social  media  data  become  available  for  analysis,  researchers  must  address  other  issues 
before  they  can  use  the  data.  Social  media  sources  inherently  raise  privacy  concerns  (Zheleva  & 
Getoor,  2009).  Storing  and  sharing  of  this  type  of  content  present  legal  concerns,  typically  arising 
from  the  "terms  of  use"  policy  for  individual  sites  by  which  users  must  abide.  Even  when  sites  allow 
data  to  be  redistributed  or  shared,  they  commonly  require  that  the  redistributed  data  be 
appropriately  modified  to  adhere  to  any  changes  made  by  the  original  author.  Aside  from  some  of 
the  better  known  issues  such  as  identity  theft,  sexual  predation,  and  employment  screening, 
continuous  concerns  arise  regarding  the  quantity  of  personal  information  most  users  share  when 
taking  part  in  various  social  media  services  (Gross  &  Acquisti,  2005).  The  terms  of  use  agreements 
established  by  the  data  providers  often  govern  subsequent  results  from  sociocultural  signature 
detection  and  forecasting.  This  can  become  quite  complicated  quickly,  as  each  provider  has  a 
separate  and  often  unique  policy  covering  the  handling,  redistribution,  and  reasoning  that  can  be 
done  on  their  data.  Unlike  many  traditional  data  sources  owned  by  various  parts  of  the 
government  or  a  small  number  of  well-known  and  longstanding  providers,  social  media  content 
providers  can  be  as  volatile  as  they  are  diverse  and  numerous. 
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Modern  advances  in  data  harvesting  technologies  enable  researchers  to  gather  traditional  and 
social  media  content  with  a  single  system,  transform  the  data  into  a  representation  that  permits 
reasoning  across  diverse  data  types,  and  perform  all  of  this  technical  work  within  the  legal  bounds 
of  a  diverse  set  of  data  providers  and  content  types.  Effective  harvesting  enables  researchers  to 
use  these  data  to  provide  validated,  consistent,  timely,  and  unique  perspectives  for  behavioral 
sensemaking  by  employing  the  modeling  and  detection  techniques  discussed  in  the  next  section. 


4.  Detecting  and  Forecasting  Behavioral  Patterns  in  Social  Media  Content 

Over  the  past  decade,  researchers  have  demonstrated  tremendous  interest  in  quantitatively 
analyzing  the  rapidly  growing  phenomenon  of  social  media  communications.  Application  areas 
include  emergency  response,  marketing,  law  enforcement  analysis  and  operations,  public  health 
tracking,  and  education  campaigns.  The  foundational  work  in  social  networks  comes  from  social 
science  research  conducted  decades  before  the  existence  of  electronic  social  media  (Rapoport, 
1957;  1963).  The  quantitative  network  science  community  (largely  physicists)  has  developed  a 
range  of  methods,  beginning  with  the  Entity-relationship  model  of  random  graphs  (Erdos  &  Renyi, 
1960),  extended  by  identification  of  the  properties  of  small-world  (Watts  &  Strogatz,  1998)  and 
scale-free  (Barabasi  &  Albert,  1999)  networks,  and  by  a  proliferation  of  empirical  and  theoretical 
studies  across  a  range  of  disciplines. 

SMSs  can  be  successfully  applied  to  social  media  data.  For  example,  Bermingham,  Conway, 
Mclnerney,  O'Hare,  &  Smeaton  (2009)  describe  an  approach  based  on  social  network  analysis  and 
sentiment  analysis  for  assessing  the  attitudes  of  members  of  a  YouTube  group  with  a  radicalizing 
agenda.  They  examine  the  topics  discussed  by  the  group  members  and  the  sentiment  polarity 
associated  with  these  topics,  as  shown  in  Figure  11.  They  then  use  the  notion  of  network  centrality 
to  assess  the  relative  importance  of  network  members. 


Concept  (No.  Documents) 


Concept  (No.  Documents) 


Figure  11.  Sentiment  analysis  results  for  females  (left  graph)  and  males  (right  graph).  Adapted  from 
Bermingham  et  al.  (2009). 


Recently,  Reynolds  et  al.  (2013)  developed  several  measures  to  look  for  signatures  of  rhetoric  used 
by  groups.  These  measures  exploit  linkage  patterns  in  networks  constructed  by  filtering  billions  of 
tweets  based  on  various  term  lists  (Fortunato,  2010;  Newman,  2003;  Porter,  Onnela,  &  Mucha, 
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2009).  Their  research  shows  that  there  is  a  strong  correlation  between  the  use  of  shared  sociolects 
(e.g.  specialized  vocabularies)  and  group  structure.  More  specifically: 

•  the  number  of  users  of  multiple  terms  drops  dramatically  as  the  number  of  terms  included 
in  the  sociolect  grows  beyond  a  given  threshold  (e.g.,  six  terms) 

•  association  in  sociolect-based  networks  (measured  in  terms  of  edge  density)  tends  to  be 
two  orders  of  magnitude  stronger  than  in  random  term-list  networks 

•  increased  term  usage  is  correlated  with  term  dependence  in  sociolect-based  networks,  but 
not  in  random  term-list  networks  where  strong  term  independence  prevails. 

Their  method  explicitly  addresses  the  issue  of  group  sizes,  which  has  proven  important  in  the 
closely  related  problem  of  community  detection  (Fortunato  &  Barthelemy,  2007;  Lancichinetti  & 
Fortunato,  2011).  Ashour  (2010)  addresses  the  question  of  online  de-radicalization  by  leveraging 
the  two  growing  phenomena  of  "online  radicalization"  and  "behavioral,  ideological,  organizational 
de-radicalization,"  as  a  foundation  on  which  to  outline  a  broad  strategy  for  countering  the 
narratives  of  violent  extremists. 

Sanfilippo  et  al.  (2013)  use  automated  frame  analysis  (Sanfilippo  et  al.  2009;  2011)  to  map  Twitter 
postings  related  to  the  Arab  Spring  into  frame-based  feature  vectors.  They  then  apply  the  feature 
probabilities  from  the  violent  intent  detection  model  discussed  in  section  2.2  (see  Figure  10)  to 
capture  ebbs  and  flows  in  sociopolitical  contention  across  geopolitical  regions  of  interest.  The 
analysis  first  aggregates  Twitter  postings  by  a  specific  time  interval  (e.g.,  days,  weeks,  months)  and 
then  maps  each  aggregated  unit  into  an  SDS.  Thereafter,  it  measures  the  contention  of  each  data 
signature  as  the  sum  of  the  probabilities  associated  with  features  promoting  violent  behavior,  as 
shown  in  (8)  where  \Ftj\  is  the  number  of  times  a  feature  /  occurs  in  message  j,  and  is  the 
probability  (or  weight)  with  which  the  feature  Ft  identifies  terrorist  rhetoric  in  the  referent  model, 
e.g.,  the  MBN  (or  decision  tree)  discussed  above. 

(8)  contention(message j)  =  Yi=i  \F..  |  *  p(Fj) 

Figure  12  shows  a  partial  graphic  representation  of  the  output  of  this  process  for  the  Twitter 
postings  about  Syria. 
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Figure  12.  Observed  levels  of  sociopolitical  contention  in  Twitter  postings  for  Syria  during  the 
period  01/10/2011  through  09/26/2011.  The  figure  shows  only  the  top  10  attributes  from  the  MBN 
model  described  in  section  2.2  that  have  a  higher  correlation  with  terrorist  messages  as  compared 
to  non-terrorist  messages. 


Overall,  strong  correlations  appear  between  peaks  of  sociopolitical  contention  and  events  that 
proved  decisive  in  charting  the  development  of  the  Arab  Spring  in  all  four  countries  for  which  the 
researchers  harvested  and  analyzed  Twitter  data.  For  example,  the  strongest  spikes  in  sociopolitical 
contention  for  the  Syrian  data  represented  in  Figure  12  (March  14-21,  2011)  coincide  with  the 
escalation  of  the  civil  conflict  in  Syria  when  security  forces  started  to  kill  protesters.  The  other 
Twitter  data  analyzed  showed  similar  correlations.  For  example,  the  highest  peaks  of  contention  in 
the  Libyan  Twitter  data  for  the  period  January-September  2011  coincide  with  the  following  events: 

•  February  2011 :  Beginning  of  protests  in  Benghazi  (2/16/11)  and  the  first  casualties  among 
protesters  (2/18/2011) 

•  March  2011 :  First  major  counterattack  by  Gaddafi  in  Zawiyah  (3/5/2011),  culminating  with 
air  attacks  on  the  civilian  population  that  led  to  Britain  and  the  United  States  implementing 
a  no-fly  zone  (3/8/2011) 

•  August  2011:  Establishment  of  the  rebel  interim  government  in  Tripoli  (8/26/11). 

This  analysis  can  also  be  applied  to  single  communication  sources  by  aggregating  time  series  data 
for  individual  users. 

5.  From  Detection  to  Prediction 

The  development  of  models  to  detect  sociocultural  signatures  promotes  sensemaking  in  two  major 
ways.  First,  it  enables  situation  awareness  through  an  understanding  of  behavioral  patterns  that 
underlie  human  motivation  and  intent,  as  described  in  the  section  above.  Second,  researchers  can 
use  the  sociocultural  signatures  that  emerge  from  the  detection  models  to  inform  prediction 
models.  For  example,  developers  can  use  the  SDSs  that  result  from  applying  detection  models  of 
violent  intent  to  time  series  data  (Figure  12)  to  train  time  series  models  that  forecast  how 
sociopolitical  contention  may  grow  and  highlight  the  indicators  responsible  for  this  growth.  With  a 
machine  learning  approach  to  time  series  forecasting  (Ahmed,  Atiya,  El  Gayar,  &  El-Shishiny,  2010), 


Current  trends  in  the  detection  of  sociocultural  signatures:  Data-driven  models  |  167 


the  same  regression  algorithms  used  for  building  classification  models  (such  as  the  decision  trees 
discussed  in  section  2)  can  help  forecast  future  time  series  from  observed  time  series,  as  shown  in 
Figure  13. 


Figure  13.  Observed  (01/11/2010  through  09/26/2011)  and  forecast  (8  weeks  ahead)  levels  of 
sociopolitical  contention  in  Twitter  postings  for  Syria.  The  figure  shows  only  the  top  10  attributes 
from  the  MBN  model  described  in  section  2.2  that  have  a  higher  correlation  with  terrorist 
messages  as  compared  to  non-terrorist  messages.  This  analysis  was  generated  using  Weka's  time 
series  forecasting  capabilities  (Weka  Time  Series). 


Forecasted  sociocultural  signatures  can  be  evaluated  using  tests  such  as  direction  accuracy  (DAC). 
DAC  measures  the  percentage  of  correct  predictions  in  direction  changes  by  counting  the  number 
of  times  that  current  and  previous  values  in  the  observed  and  predicted  time  series  move  in  the 
same  direction,  as  shown  in  (9).  Figure  14  shows  the  direction  accuracy  for  the  eight  predicted 
week  steps  in  the  time  series  shown  in  Figure  13. 

(9) 

where:  at  =  1,  if  sgnfcurrenti  —  previousi )  =  sgn(current *  -  previous *) 
at=  0  otherwise, 

and  current* ,  previous* are  predicted  values. 


Figure  14.  Direction  accuracy  evaluation  for  the  eight  predicted  week  steps  in  the  time  series 
shown  in  Figure  13. 
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The  reliability  of  prediction  models  such  as  the  time  series  forecasts  described  above  depends  on 
the  quality  and  amount  of  the  input  data,  as  determined  by  the  signature  detection  approaches 
adopted.  Modeling  techniques  that  require  fewer  training  data  points,  such  as  the  application  of 
support  vector  regression  to  time  series  forecasting  (Muller,  Smola,  Ratsch,  Scholkopf,  Kohlmorgen, 
&  Vapnik,  1997)  and  smoothing  methods  to  correct  modeling  inaccuracies  due  to  data  sparseness 
(Zhai  &  Lafferty,  2001),  can  partly  address  these  challenges.  Ultimately,  the  availability  of  better 
capabilities  to  harvest  social  media  content  and  more  advanced  methods  for  the  detection  of 
sociocultural  signatures  will  determine  the  ability  to  develop  more  reliable  prediction  models. 

6.  Conclusions 

The  extraction  of  SDSs  from  document  repositories  and  their  use  in  developing  models  that  detect 
sociocultural  signatures  form  the  basis  of  sociocultural  behavior  sensemaking.  Applying  methods 
for  sociocultural  signature  detection  to  social  media  generates  novel  and  effective  intelligence  for 
situation  awareness  and  decision  making,  and  in  turn  leads  to  improved  sensemaking  through  the 
analysis  of  interactive  communication  on  social  media  channels.  In  the  next  few  years  we  can 
expect  significant  advances  in  understanding  and  predicting  the  course  of  individual  and  collective 
behaviors  such  as  sociopolitical  contention  and  the  propensity  to  engage  in  violent  behavior. 

Needless  to  say,  the  endeavor  we  have  described  presents  no  shortage  of  challenges  and 
opportunities  with  regard  to  technical  issues  (e.g.,  harvesting  and  aggregation,  modeling),  legal 
concerns  (e.g.,  privacy  protection),  and  reliability  (e.g.,  provenance-based  trust,  data  sparseness). 
Data  collection  and  analysis  have  become  both  increasingly  complex  and  increasingly  rewarding 
due  to  the  expansion  of  social  media  content.  Authorship  in  English— a  ripe  practice  in  social 
media— and  manual  or  automated  translation  of  non-English  texts  reduce  linguistic  bottlenecks, 
but  we  still  lack  clear  understanding  of  how  the  loss  of  cultural  nuances  affects  data  analysis  when 
working  with  non-native  text.  The  application  of  computer  modeling  to  communicative  processes 
strongly  augments  our  ability  to  understand  and  anticipate  human  interaction  in  diverse 
sociocultural  contexts,  but  the  field  of  content  analysis  remains  too  fragmented  to  provide  a 
comprehensive  approach  to  the  study  of  human  communication. 

Finally,  while  computational  methods  have  greatly  aided  analysis  and  detection  tasks,  the 
combination  of  human  and  machine  intelligence  remains  an  open  issue.  The  development  of 
collaborative  working  environments  in  which  analysis  and  decision  making  emerge  from  the 
concerted  articulation  of  human-to-human,  human-to-machine,  and  machine-to-machine 
interactions  is  still  in  its  early  stages  and  only  ready  for  operational  use  in  very  limited  forms. 
Successful  resolution  of  these  issues  or  mitigation  of  the  ensuing  challenges  to  reap  the  associated 
opportunities  will  largely  determine  the  success  of  this  emerging  analytic  endeavor. 
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1.  Introduction 

To  discover,  distinguish,  and  locate  operationally  relevant  sociocultural  signatures,  no  tool 
surpasses  the  human  eye  applied  to  appropriate  displays  of  relevant  data.  Indeed,  the  human  brain 
generally  outperforms  computer-based  algorithms  at  finding  patterns  in  data,  particularly  patterns 
that  are  a  priori  ill-specified  or  unknown.  For  confirmation  of  this,  one  need  look  no  further  than 
the  captchas2  used  on  webpage  login  screens. 

Visualization  is  the  science— and  the  art— of  discovering  trends  and  patterns  in  data.  Those  data 
can  be  quantitative  or  qualitative;  they  can  be  temporal,  spatial,  both,  or  neither.  The  appropriate 
visualization  of  the  data  allows  the  viewer  to  move  from  the  specific  to  the  general,  ideally  gaining 
some  insight  into  the  larger  realm  from  which  the  data  came,  perhaps  along  with  some  insight  into 
the  underlying  social,  cultural,  or  other  phenomena  of  interest.  As  such,  the  appropriate 
visualization  is  often  specific  to  the  particular  problem  or  research  question  at  hand  and  to  the 
available  data. 

Done  correctly,  data  visualization  can  facilitate: 

•  Insight :  Data  visualization  can  provide  insights,  particularly  into  relationships  among 
variables  and  trends  over  time,  that  may  not  be  apparent  otherwise.  Appropriate 
visualizations  can  make  large  and/or  complicated  data  easier  to  understand. 

•  Exploration :  Interactive  visualization  methods  permit  analysts  to  assess  data  from  multiple 
perspectives  and  contexts;  these  methods  also  allow  sociocultural  experts  to  bring  their 
expertise  to  bear  on  the  visualization  and  ultimate  interpretation  of  the  data. 

•  Impact :  "Use  a  picture.  It's  worth  a  thousand  words."3  Data  visualization  is  a  powerful  way 
to  communicate  information,  relationships,  and  even  the  stories  present  in  data. 

Data  visualization  can  reveal  relationships  that  summary  statistics  simply  cannot  convey.  The 
canonical  example  is  Anscombe's  data,  plotted  in  Figure  1  (Anscombe,  1973).  Visually  these  four 
sets  of  data  clearly  show  very  different  relationships  between  the  x  and  y  variables,  yet  the  means 

1  Approved  for  Public  Release;  Distribution  Unlimited.  Case  Number  14-2487 
Copyright  ©  2014  The  MITRE  Corporation. 

2  Captcha  is  short  for  Completely  Automated  Public  Turing  Test  To  Tell  Computers  and  Humans  Apart.  See 
http://www.captcha.net. 

3  Arthur  Brisbane,  "Speakers  Give  Sound  Advice,"  Syracuse  Post  Standard  (page  18),  March  28, 1911. 
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and  standard  deviations  of  each  of  the  x  variables  are  exactly  the  same;  the  means  and  standard 
deviations  of  the  y  variables  are  also  the  same;  the  correlations  between  x  and  y  in  each  of  the  four 
cases  are  the  same;  and,  even  the  regression  fits  are  the  same.  Merely  examining  some  summary 
statistics  without  plotting  the  data  could  completely  mislead  a  viewer  into  believing  that  there  is 
little  difference  in  the  underlying  phenomena. 


xl 


x2 


x3 


x4 


Figure  1.  Plots  of  Anscombe's  data,  where  all  the  summary  statistics  match,  the  regression  fits  (as 
shown  by  the  lines)  are  the  same,  and  yet  the  relationships  between  the  x  and  y  variables  are 
clearly  different  in  all  four  cases. 


In  terms  of  communication  and  collaboration,  Sviokla  (2009)  states  that  visualization  brings  the 
following  advantages: 

•  Great  visualizations  are  efficient— they  let  people  look  at  vast  quantities  of  data  quickly. 

•  Visualizations  can  help  an  analyst  or  a  group  achieve  more  insight  into  the  nature  of  a 
problem  and  discover  new  understanding. 

•  A  great  visualization  can  help  create  a  shared  view  of  a  situation  and  align  folks  on  needed 
actions. 
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As  Henry  Hubbard  said  in  Brinton's  1939  text,  Graphic  Presentation, 

There  is  a  magic  in  graphs.  The  profile  of  a  curve  reveals  in  a  flash  a  whole  situation  -  the  life 
history  of  an  epidemic,  a  panic,  or  an  era  of  prosperity.  The  curve  informs  the  mind,  awakens 
the  imagination,  convinces  (Brinton,  1939,  p.  2). 

In  summary,  using  visualization  has  clear  benefits,  and  these  benefits  increase  in  direct  relation  to 
the  size  of  the  data,  although  even  with  small  data  sets  such  as  Anscombe's  visualization  can 
provide  unique  advantages.  But  unlike  Anscombe's  data,  which  is  small  and  simple  to  visualize  via 
scatterplots,  sociocultural  data  can  be  large,  complicated,  and  messy.  Thus,  the  real  question 
concerns  how  one  can  appropriately  visualize  such  data. 

1.1.  Visualization  for  Sociocultural  Signature  Detection 

As  defined  in  the  Office  of  the  Secretary  of  Defense  (OSD)  Human  Social  Culture  Behavior  (HSCB) 
Modeling  program,  sociocultural  signature  detection  is  one  of  the  four  program  capabilities: 
Understand,  Detect,  Forecast,  and  Mitigate.  The  Detect  capability  consists  of  "capabilities  to 
discover,  distinguish,  and  locate  operationally  relevant  sociocultural  signatures  through  the 
collection,  processing,  and  analysis  of  sociocultural  behavior  data"  (Office  of  the  Secretary  of 
Defense,  2013).  The  site  goes  on  to  say, 

Once  the  defining  features  of  the  sociocultural  setting  are  understood,  the  next  steps  are  to 
develop  a  persistent  capability  to  detect  sociocultural  behavior  signals  of  interest  amidst 
complexity  and  noise,  and  to  harvest  data  for  analysis.  This  entails  capabilities  for  ISR  in  the 
area  of  sociocultural  behavior  (referred  to  here  as  a  "social  radar"),  with  particular  focus  on  the 
challenges  associated  with  open  source  data  collection.  It  also  requires  robust  systems  for 
storing  and  managing  that  data,  and  tools  enabling  timely,  dynamic  analysis. 

Visualization,  then,  supports  "enabling  timely,  dynamic  analysis"  in  terms  of  both  finding  relevant 
signatures  and  identifying  when  known  signatures  change.  The  former  is  largely  a  retrospective 
exercise  in  exploring  and  modeling  existing  data  to  identify  sociocultural  signatures,  while  the  latter 
is  a  prospective  exercise  in  monitoring  a  given  signature  to  identify  if  and  when  it  changes.  Both 
types  of  analysis  may  require  a  variety  of  cross-sectional,  temporal,  and  spatiotemporal  analytical 
methods  and  visualization  techniques  for  infrastructure,  social,  and  other  types  of  network  data; 
Geographic  Information  System  (GIS)  and  similar  types  of  spatial  data;  surveys  and  related  types  of 
data;  and  social  media  and  other  types  of  linguistic  data.4 

Good  visualization  for  sociocultural  signature  detection  must  be  optimized  for  the  particular 
application  and  user.  For  example,  the  best  methods  for  visualizing  networks  may  be  quite 
different  than  those  for  displaying  GIS  data.  Similarly,  GIS  methods  for  displaying  point  data  on 
maps  are  not  appropriate  for  displaying  areal  data  from  surveys.  Some  types  of  social  media  data 


4  Visualization  is  equally  important  and  relevant  for  the  Understand  capability.  In  fact,  there  is  no  clear 
dividing  line  between  the  two  capabilities  in  terms  of  data  visualization.  Many  of  the  visualization  techniques 
are  the  same;  the  main  distinction  lies  in  how  the  methods  are  employed.  See  Chapter  Three 
(Understand/Visualization)  in  this  book  for  additional  discussion. 
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can  be  effectively  displayed  as  a  network,  while  other  types  require  different  visualization 
methods. 

Tufte  (1997,  2001,  2006),  Cleveland  (1993,  1994)  and  others  have  addressed  the  general  question 
of  effective  design  for  quantitative  visualization.  For  example,  Tufte  (2001,  p.  13)  says, 

Excellence  in  statistical  graphics  consists  of  complex  ideas  communicated  with  clarity,  precision, 
and  efficiency.  Graphical  displays  should 

•  show  the  data 

•  induce  the  viewer  to  think  about  the  substance  rather  than  about  methodology,  graphic 
design,  the  technology  of  graphic  production,  or  something  else 

•  avoid  distorting  what  the  data  have  to  say 

•  present  many  numbers  in  a  small  space 

•  make  large  data  sets  coherent 

•  encourage  the  eye  to  compare  different  pieces  of  data 

•  reveal  the  data  at  several  levels  of  detail,  from  the  broad  overview  to  the  fine  structure 

•  serve  a  reasonably  clear  purpose:  description,  exploration,  tabulation,  or  decoration 

•  be  closely  integrated  with  the  statistical  and  verbal  descriptions  of  the  data  set 

Of  course,  the  devil  is  in  the  details:  How  can  a  visualization  method  meet  these  criteria  for  a 
specific  sociocultural  analytical  method  in  a  given  scenario  for  a  particular  analyst  or  researcher? 

1.2.  Visualization  as  Part  of  Data  Exploration 

Exploratory  data  analysis  or  EDA  (Tukey,  1977)  focuses  on  summarizing  data  to  make  it  easy  to 
understand,  often  through  graphics  and  data  displays,  and  generally  without  using  formal  statistical 
models  or  hypotheses.  As  described  by  the  National  Institute  of  Standards  and  Technology  (NIST, 
2012): 

Exploratory  Data  Analysis  (EDA)  is  an  approach/philosophy  for  data  analysis  that  employs  a 
variety  of  techniques  (mostly  graphical)  to 

•  maximize  insight  into  a  data  set; 

•  uncover  underlying  structure; 

•  extract  important  variables; 

•  detect  outliers  and  anomalies; 

•  test  underlying  assumptions; 

•  develop  parsimonious  models;  and 

•  determine  optimal  factor  settings. 


NIST  goes  on  to  say, 

The  particular  graphical  techniques  employed  in  EDA  are  often  quite  simple,  consisting  of 
various  techniques  of: 
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•  Plotting  the  raw  data  (such  as  data  traces,  histograms,  bihistograms,  probability  plots, 
lag  plots,  block  plots,  and  Youden  plots). 

•  Plotting  simple  statistics  such  as  mean  plots,  standard  deviation  plots,  box  plots,  and 
main  effects  plots  of  the  raw  data. 

•  Positioning  such  plots  so  as  to  maximize  our  natural  pattern-recognition  abilities,  such 
as  using  multiple  plots  per  page. 


Hence,  EDA  is  a  methodology  for  learning  from  data,  often  using  visualization.  The  discussion  in  this 
chapter  emphasizes  the  visual  display  of  data  in  support  of  information  exploration  for  detection  of 
sociocultural  signatures.  Some  may  take  issue  with  either  an  over-reliance  on  more  informal 
exploratory  methods  or  a  failure  to  use  confirmatory  methods;  however,  as  Tukey  (1977,  p.  vii) 
said, 

Once  upon  a  time,  [researchers]  only  explored.  Then  they  learned  to  confirm  exactly-to  confirm 
a  few  things  exactly,  each  under  very  specific  circumstances.  As  they  emphasized  exact 
confirmation,  their  techniques  inevitably  became  less  flexible.  The  connection  of  the  most  used 
techniques  with  past  insights  has  weakened.  Anything  to  which  a  confirmatory  procedure  was 
not  explicitly  attached  was  decried  as  "mere  descriptive  statistics",  no  matter  how  much  we 
have  learned  from  it. ...  Today,  exploratory  and  confirmatory  can-and  should-proceed  side  by 
side.  [Emphasis  in  the  original  text.] 


Particularly  in  terms  of  exploring  data  to  detect  sociocultural  signatures,  EDA  is  an  inherently 
interactive  exercise.  As  Yi,  ah  Kang,  Stasko,  and  Jacko  (2007,  p.  1224)  say, 

Interaction  is  an  essential  part  of  Infovis  [information  visualization],  however.  Without 
interaction,  an  Infovis  technique  or  system  becomes  a  static  image  or  autonomously  animated 
images  (e.g.,  InfoCanvas).  While  static  images  clearly  have  analytic  and  expressive  value,  their 
usefulness  becomes  more  limited  as  the  data  set  that  they  represent  grows  larger  with  more 
variables. 


1.3.  Caution:  Apophenia  and  Pareidolia 

Shermer  (2008)  noted  that,  "...our  brains  are  belief  engines:  evolved  pattern-recognition  machines 
that  connect  the  dots  and  create  meaning  out  of  the  patterns  that  we  think  we  see  in  nature. 
Sometimes  A  really  is  connected  to  B;  sometimes  it  is  not."  Thus,  while  the  human  brain  excels  at 
finding  true  patterns  against  noisy  backgrounds,  it  also  quite  capable  of  over-interpreting  pure 
noise  to  find  nonexistent  "patterns."  This  phenomenon  of  finding  (seemingly)  meaningful  patterns 
in  random  or  meaningless  data  is  called  apophenia;  pareidolia  is  visual  apophenia  (Hoopes,  2011). 
To  anyone  who  has  spent  a  summer  afternoon  looking  for  animal  shapes  in  the  clouds,5  the  act  of 


5  As  Shakespeare  (1936)  wrote  in  Hamlet :  "Hamlet:  Do  you  see  yonder  cloud  that's  almost  in  shape  of  a 
camel?  Polonius:  By  the  mass,  and  'tis  like  a  camel,  indeed.  Hamlet:  Methinks  it  is  like  a  weasel.  Polonius:  It  is 
backed  like  a  weasel.  Hamlet:  Or  like  a  whale?  Polonius:  Very  like  a  whale." 


178  |  Visualization  for  sociocultural  signature  detection 


then  believing  the  shapes  represent  something  other  than  the  coincidental  arrangement  of  water 
vapor  in  the  upper  atmosphere  is  an  example  of  pareidolia.6  As  Silver  (2012,  p.  240)  says,  "Finding 
patterns  is  easy  in  any  kind  of  data-rich  environment...  The  key  is  in  determining  whether  the 
patterns  represent  noise  or  signal." 

In  statistical  terms,  apophenia  and  pareidolia  are  Type  I  errors:  the  false  positive  identification  of 
"patterns"  in  data.  This  type  of  error  is  non-trivial  in  sociocultural  analyses,  particularly  since  an 
assumption  underlying  much  of  the  visualization  literature  seems  to  be  that  a  true  pattern  exists  in 
the  data  and  that  the  main  question  is  how  to  best  display  that  pattern.  In  contrast,  exploratory 
analyses  of  data  in  which  one  looks  for  patterns  that  may  or  may  not  be  present  always  have  the 
potential  for  false  positives.  Under  these  conditions,  it  is  important  to  recall  the  human  ability  to 
find  fanciful  camels  in  the  clouds,  and  then  readily  to  construct  an  a  posteriori  rationale  explaining 
the  existence  and  meaning  of  the  camel.  But  it  is  equally  important  to  exploit  the  human  facility  for 
pattern  recognition,  particularly  with  complex  sociocultural  signature  data.  Thus,  one  challenge 
with  exploratory  data  visualization  and  pattern  recognition  is  to  balance  human  and 
methodological  sensitivity  for  recognizing  true  patterns  against  the  over-sensitivity  that  results  in 
false  positive  "patterns." 


2.  Visualization  to  Detect  Sociocultural  Signatures 

In  this  section  we  first  review  some  of  the  classical  methods  and  approaches  for  displaying  data. 
We  then  move  on  to  discuss  visualization  methods  and  techniques  in  the  context  of  fairly  general 
problem  and  data  classes  —  networks,  geographically  based  data,  surveys,  linguistic  data,  and 
social  media  —  that  are  particularly  relevant  to  sociocultural  signature  detection. 

2.1.  Classical  Approaches  to  Data  Visualization 

Data  visualization  has  historically  been  thought  of  in  terms  of  graphing  numerical  data.  Typically 
these  data  were  either  cross-sectional  (i.e.,  collected  at  the  same  point  in  time)  or  longitudinal  (i.e., 
collected  over  time)  and  were  either  continuous  or  discrete,  with  different  methods  developed  to 
address  each  combination.  For  example,  typical  graphical  methods  for  continuous  cross-sectional 
data  include  histograms  and  box  plots,  while  methods  for  discrete  cross-sectional  data  include  a 
variety  of  bar  and  pie  charts,  as  well  as  simple  tabular  summaries.  Longitudinal  data  are  most 
typically  plotted  on  some  type  of  time  series  plot. 

Visualization  methods  for  two  variables  include  scatterplots,  mosaic  plots,  side-by-side  box  plots, 
and  others.  The  scatterplot  matrix  (also  known  as  a  pairs  plot)  permits  simultaneous  presentation 
of  multiple  scatterplots.  It  is  typically  displayed  in  a  square  showing  n  continuous  variables  on  both 
the  vertical  and  horizontal  axes,  where  each  row  and  each  column  depicts  one  variable's 
comparison  with  the  other  n  -  1  variables.  The  diagonal  is  typically  left  for  labels  or  single-variable 
graphs  such  as  histograms,  since  no  meaning  can  be  derived  from  a  scatterplot  that  compares  a 


6  A  somewhat  related  phenomenon  is  the  cherry  picking  of  facts  to  suit  a  preconceived  notion.  As  Sherlock 
Flolmes  warns,  "It  is  a  capital  mistake  to  theorize  before  one  has  data.  Insensibly  one  begins  to  twist  facts  to 
suit  theories,  instead  of  theories  to  suit  facts"  (Doyle,  2003,  p.  189). 
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variable  to  itself.  The  upper  triangle  of  the  square  displays  the  same  information  as  the  lower 
triangle,  albeit  inverted  (i.e.,  "X  versus  Y”  translates  to  "Y  versus  X”),  so  it  is  not  uncommon  to 
display  only  one  side  of  the  diagonal.  All  benefits  of  the  scatterplot  (e.g.,  visualizing  correlation)  are 
inherited,  though  the  scatterplot  matrix  requires  the  viewer  to  cognitively  examine  and  potentially 
explore  each  pair  of  variables  individually. 

Tukey  (1977)  and  Cleveland  (1993;  1994)  give  a  more  detailed  discussion  of  classic  data 
visualization  methods.  These  and  more  recent  methods,  described  in  the  context  of  modern 
statistical  software,  are  discussed  in  texts  such  as  Wilkinson,  Wills,  Rope,  Norton,  and  Dubbs 
(2005),  Wickham  (2009),  Sarkar  (2008),  and  Murrell  (2011),  although  the  texts  still  tend  to  focus  on 
statistical  and  quantitative  graphics.  Discussions  of  visualization  from  a  media  and/or  graphical 
design  perspective  include  Yau  (2011;  2013),  Steele  and  lllinsky  (2010),  and  Wong  (2010),  in 
addition  to  the  well-known  books  by  Tufte  (1986;  1990;  1997;  2006). 

Common  to  most  of  these  methods  and  most  of  these  discussions  is  that  the  data  involved  are 
quantitative,  typically  of  low  dimension  (mainly  one  or  two,  though  sometimes  three  dimensions), 
and  the  plots  are  mainly  static.  Stated  another  way,  the  classical  data  visualization  methods  are 
generally  not  designed  for  text-based  data,  photo  and  video  data,  high-dimensional  data,  and 
geospatial  data,  all  of  which  are  becoming  more  common  at  an  increasingly  greater  rate. 

Text,  photo,  and  video  data  are  often  displayed  in  small  multiples  or  facets,  often  called  lattice  or 
trellis  plots.  These  are  particularly  useful  for  looking  at  plots  of  data  conditioned  on  the  value  of 
one  or  more  categorical  variables.  For  example,  Figure  2  is  a  lattice  plot  of  the  age  of  patients 
presenting  at  a  health  clinic  conditioned  on  gender  (male  or  female)  and  ethnicity  (unknown, 
Hispanic,  and  non-Flispanic).  The  plot  shows  clear  differences  in  the  age  distribution  of  patients 
presenting  by  gender  but  no  difference  by  ethnicity. 
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Figure  2.  Trellis  plot  example  showing  differences  in  the  distribution  of  health  clinic  patient  age  by 
gender  but  no  difference  by  ethnicity.  Adapted  with  permission  from  Fricker  (2013). 


Rosling  (2013)  displays  multiple  variables  via  bubbles  on  a  scatterplot.  In  addition  to  the  variables 
represented  on  the  two  axes,  the  size  of  the  bubble  represents  a  third  variable  and  (optionally) 
bubble  color  represents  a  fourth.  His  software,  Gopminder  World,  provides  over  500  variables  for 
comparison;  Figure  3  shows  an  example  output.  The  same  data  can  also  be  displayed 
geographically,  as  in  Figure  4,  where  the  two  axis  variables  from  Figure  3  (life  expectancy  and 
income  per  person)  are  exchanged  for  country  centroid  latitude  and  longitude,  while  the  bubble 
size  and  color  retain  their  original  meanings. 

Tufte  (2001)  cautions  against  under-  or  more  commonly  over-representing  the  magnitude  of  the 
data  when  using  area  to  display  a  variable  (the  mistake  is  not  made  in  Rosling's  work).  As  Tufte  says 
(2001,  p.  57),  "The  representation  of  numbers,  as  physically  measured  on  the  surface  of  the  graphic 
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itself,  should  be  directly  proportional  to  the  quantities  represented/'  Tufte  proposes  a  metric  called 
the  Lie  Factor,  which  is  the  ratio  of  the  size  of  the  effect  shown  in  the  graphic  and  the  size  of  the 
effect  in  the  data.  Values  within  0.95  and  1.05  are  preferred,  while  ratios  outside  of  that  range 
show  "substantial  distortion,  far  beyond  minor  inaccuracies  in  plotting"  (Tufte,  2001,  p.  57). 


400  1  000  2  000  4QOO  10000 


Income  per  person  (GDP/capita,  PPP$  inflation -adjusted)  *  log  v 

Figure  3.  Rosling's  Gapminder  World  software  provides  easy-to-interpret  displays  of  up  to  four 
variables,  with  a  fifth  presented  in  a  time-series  animation.  Dimensions  are  indicated  by  the  two 
axes  plus  the  bubble  size  and  color.  Adapted  with  permission  from  Gapminder  World  software 
display,  Rosling  (2013). 
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Figure  4.  The  two  axis  variables  from  Figure  3  are  exchanged  for  country  centroid  latitude  and 
longitude.  The  bubble  size  and  color  retain  their  relevance.  Adapted  with  permission  from 
Gopminder  World  software  display,  Rosling  (2013). 


Figure  5.  Multivariate  visualization  techniques  evaluated  in  the  study.  From  left  to  right:  brush 
strokes,  data  driven  spots,  oriented  slivers,  color  blending,  and  attribute  blocks.  Adapted  with 
permission  from  Livingston  &  Decker  (2012). 


An  individual  scatterplot  may  be  enhanced  to  display  one  or  more  additional  variables.  For 
example,  varying  the  dot  size,  color,  and/or  shape  makes  it  possible  to  add  extra  variables  to  the 
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graph.  LOESS  (locally  weighted  scatterplot  smoothing)  curves,  developed  by  Cleveland  &  Devlin 
(1988),  do  not  add  a  dimension  but  may  provide  insight  into  the  relationship  between  variables. 

Healey,  Kocherlakota,  Rao,  Mehta,  and  St.  Amant  (2008)  related  data  display  to  human  perception 
and  automated  search  strategies,  providing  a  pathway  from  the  data  to  visual  interpretation.  They 
used  luminance,  hue,  size  (height),  density,  orientation,  and  regularity  to  a  grid  to  display  high¬ 
dimensional  data.  Livingston  et  al.  (2011;  2012;  2013)  and  Livingston  &  Decker  (2012;  2011) 
continued  their  work,  intending  to  display  up  to  ten  variables  simultaneously  on  two-dimensional 
plots.  Figure  5  shows  five  of  their  techniques. 

Perhaps  one  of  the  most  elegant  portrayals  of  multidimensional  data  is  Charles  Minard's  flow  map 
of  Napoleon's  March  to  Moscow,  shown  in  Figure  6.  Tufte  (2001,  p.  40)  credits  this  graph  as 
displaying  "six  variables:  the  size  of  the  army,  its  location  on  a  two-dimensional  surface,  direction 
of  the  army's  movement,  and  temperature  on  various  dates  during  the  retreat  from  Moscow.  ...  It 
may  well  be  the  best  statistical  graphic  ever  drawn." 


Figure  6.  Classic  graphic  from  Charles  Minard  (1781-1870)  showing  the  progress  and  fate  of 
Napoleon's  army  in  its  march  against  Moscow.  This  is  a  combination  of  a  data  map  and  a  time- 
series,  and  displays  six  variables.  Adapted  with  permission  from  Tufte  (2001). 


2.2.  Network  Visualization 

Networks  occur  naturally  in  many  situations.  Conceptually  they  are  quite  simple,  with  every  entity 
denoted  by  a  node  in  the  network  and  linkages  between  entities  denoted  by  arcs.  Visualizing 
networks  in  ways  useful  for  learning  about  and  understanding  the  network,  on  the  other  hand,  can 
be  anything  but  simple.  For  example,  particularly  with  large  networks,  simply  displaying  all  the  arcs 
and  nodes  results  in  the  classic  "hairball"  (see,  for  example,  Figure  7)  from  which  users  can  discern 
little  about  the  network.  As  Krzywinski  (2013)  says, 
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"Hairballs  turn  complex  data  into  visualizations  that  are  just  as  complex,  or  even  more  so. 
Hairballs  can  even  seduce  us  to  believe  that  they  carry  a  high  information  value.  But,  just 
because  they  look  complex  does  not  mean  that  they  can  communicate  complex  information. 
Hairballs  are  the  junk  food  of  network  visualization  -  they  have  very  low  nutritional  value, 
leaving  the  user  hungry." 


Figure  7.  A  network  visualization  of  a  Facebook  friendship  network  for  15,000  users.  A  classic 
"hairball"  that  provides  little  insight  into  the  network  structure.  Adapted  with  permission  from 
Coscia  (2012). 


At  issue  is  that  "Visualizations  are  useful  to  leverage  the  powerful  perceptual  abilities  of  humans, 
but  overlapping  links  and  illegible  labels  of  nodes  often  undermine  this  approach"  (Perer  & 
Shneiderman,  2006,  p.  693).  As  a  result,  when  visualizing  networks  it  is  often  important  either  to 
execute  the  visualization  in  interactive  software  so  that  the  user  can  explore  the  network 
appropriately  and/or  to  subset  or  highlight  the  data  so  that  structure  is  visible.  Figure  8  shows  an 
example  of  the  latter  approach  in  which  the  most  connected  nodes  and  associated  arcs  are 
highlighted. 
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Figure  8.  An  example  of  a  filtered  network  that  highlights  the  most  connected  nodes  and  their 
connections  to  one  another.  Adapted  with  permission  from  Smith  et  al.  (2009). 


There  are  numerous  ways  to  visualize  networks.  Figure  9  illustrates  a  few  by  Krzywinski  (2013) 
using  hive  plot  software.  What  should  be  evident  is  that  the  most  relevant  visualization  or 
visualizations  is/are  tied  to  the  specific  problem  at  hand  and  the  associated  data.  Furthermore, 
finding  the  preferred  visualization  may  require  multiple  attempts;  software  that  allows  interactive 
exploration  and  visualization  of  the  network  facilitates  the  search  for  the  best  solution. 

Interactive  network  visualization  is  very  much  in  the  spirit  of  Tukey's  exploratory  data  analysis  and 
in  keeping  with  recommendations  by  Perer  and  Shneiderman  (2006,  p.  40),  who  promote  an 
"overview  first,  then  details  on  demand"  approach  to  visualization.  Similarly,  Viegas  and  Donath, 
(2004,  p.  1)  say,  "We  posit  that  basic  cartographic  principles  -  such  as  adaptive  zooming  and 
multiple  viewing  modes  -  provide  system  designers  with  useful  visual  solutions  to  the  depiction  of 
social  networks." 

The  variety  and  quantity  of  network  visualization  approaches  and  software  are  too  great  to  permit 
a  comprehensive  review  in  the  limited  space  available  in  this  chapter.  Furthermore,  the  practice  is 
undergoing  rapid  development,  so  any  listing  will  quickly  become  outdated.  Instead,  readers 
should  consult  on-line  resources  such  as  Mobio  (2013)  for  an  overview.  As  an  interesting  example 
of  self-referential  visualization,  Figure  10  shows  a  screenshot  of  an  interactive  network 
visualization  of  data  visualization  resources. 
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Figure  9.  Alternative  network  visualizations:  (a)  normalized  (top)  and  absolute  (bottom) 
connectivity;  (b)  co-regulation  networks;  (c)  network  edges  shown  as  ribbons  creating  circularly 
composited  stacked  bar  plots  (a  periodic  streamgraph);  (d)  syntenic  network;  and  (e)  layered 
network  correlation  matrix.  Adapted  with  permission  from  Krzywinski  (2013). 
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Figure  10.  Interactive  network  visualization  of  data  visualization  resources.  This  particular  view 
highlights  visualization  software.  Adapted  with  permission  from  Mobio  (2013). 
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2.3.  Visualization  of  Geographic  Information 

The  visualization  of  geographic  data  -  the  making  of  maps  -  has  a  long  history  and  recent 
developments  in  computing  have  put  the  display  of  geographic  data  within  easy  reach. 
Furthermore,  the  widespread  incorporation  of  Global  Positioning  System  (GPS)  capabilities  in 
objects  such  as  mobile  phones  and  freight  palettes  has  produced  significant  growth  in  the  amount 
of  georeferenced  data  available  to  consumers  and  organizations.  Additionally,  it  is  now 
straightforward  to  perform  analyses,  as  distinct  from  visualizations,  that  would  have  been  much 
more  difficult  even  ten  years  ago.  By  analysis,  we  refer  here  to  tasks  such  as  identifying  the 
polygons  that  contain  particular  points  (for  example,  identifying  terrorist  incidents  with  provinces), 
projecting  points  onto  lines  (for  example,  identifying  traffic  accidents  with  road  locations),  or 
computing  lines  of  sight  (for  example,  locating  regions  that  cannot  be  seen  from  a  particular 
tower).  Of  course,  the  distinction  between  display  and  analysis  is  not  always  sharp. 

2.3.1.  Coordinate  systems 

One  aspect  that  sets  geographic  visualization  apart  from  traditional  approaches  is  that  developers 
must  choose  the  coordinate  systems  with  care.  At  the  start,  data  are  often  associated  with 
geographic  coordinates— latitude  and  longitude,  typically  in  degrees.  Since  the  earth  is  roughly 
ellipsoidal,  geographic  coordinates  must  be  associated  with  a  "datum"  that  characterizes  the 
particular  ellipsoidal  approximation  in  use.  A  latitude,  longitude  pair  for  a  particular  location 
derived  under  one  datum  might  be  hundreds  of  meters  away  from  the  same  pair  derived  under 
another  datum.  A  number  of  datums7  are  available,  but  the  WGS84  datum,  used  by  GPS,  is  by  far 
the  most  common.  Several  computer  programs  make  it  possible  to  convert  coordinates  based  on 
one  datum  to  another. 

Over  small  areas  geographic  coordinates  can  be  plotted  directly,  neglecting  the  curvature  of  the 
earth,  but  for  even  moderate  distances  (say,  dozens  of  miles  or  scores  of  kilometers)  a  projection  is 
necessary.  As  one  interesting  example,  the  two  towers  of  New  York's  Verrazano-Narrows  bridge 
are  1-5/8  inches  farther  apart  at  the  top  than  at  the  bottom  because  of  the  curvature  over  the 
bridge's  4,260-foot  length. 

Dozens  of  projections  are  available,  with  different  projections  serving  different  roles.  For  example, 
the  widely  used  transverse  Mercator  projection  preserves  angles— that  is,  angles  on  the  map 
match  angles  on  the  ground— and  is  therefore  useful  for  navigation.  The  Albers  projection  shows 
areas  accurately,  which  might  be  particularly  useful  in  displaying  political  data.  Figure  11  shows  the 
lower  48  states  of  the  United  States  under  the  Mercator  (upper)  and  Albers  (lower)  projections, 
together  with  a  grid  of  lines  of  constant  longitude  and  of  constant  latitude.  Any  text  about  GIS  (for 
example,  Bolstad,  2008)  lists  and  compares  the  common  projections. 

Projected  coordinates  are  often  displayed  as  latitudes  and  longitudes,  but  sometimes  they  are 
converted  to  UTM,  the  Universal  Transverse  Mercator  grid  system  (or  perhaps  its  military  analog, 
the  Military  Grid  Reference  System).  UTM  represents  the  globe  by  a  set  of  sixty  projections,  each 
six  degrees  of  longitude  wide  by  eight  degrees  of  latitude  high.  Longitude  bands  are  numbered; 


7  In  the  geodetic  context,  the  plural  of  "datum"  is  always  "datums." 


Visualization  for  sociocultural  signature  detection  |  189 


latitude  bands  are  lettered.  Within  a  zone,  points  are  labeled  with  "eastings"  and  "northings"  in 
meters.  This  makes  it  easy  to  compute  lengths  and  areas  directly  from  the  coordinates;  within  a 
zone  the  distortion  in  a  length  computed  this  way  is  held  under  0.1%.  Computing  distances 
between  points  in  different  zones,  however,  requires  a  more  difficult  computation. 


Figure  11.  Two  projections  of  the  lower  48  states  of  the  U.S.A.  The  upper  shows  the  familiar 
Mercator  projection,  the  lower  the  equal-area  Albers  projection. 


2.3.2.  Tools  and  formats  for  geographic  data 

The  final  product  of  a  visualization  is  generally  a  graphic  image  in  digital  form.  Geographic  data  can 
be  held,  manipulated,  and  depicted  in  a  large  number  of  formats.  Often  one  piece  of  software 
saves  the  data  in  projected  form,  which  means  that  the  data  may  have  to  be  both  re-projected  and 
converted  to  be  used  in  another  software  system. 

Many  sets  of  tools,  often  known  as  GIS,  are  available  for  displaying  geographic  data.  Among  the 
most  popular  are  the  ArcGIS  suite  from  Esri  Corporation,  which  is  powerful  and  comparatively 
expensive;  the  free,  open-source  GRASS  [Geographic  Resources  Analysis  Support  System]  GIS  from 
the  Open  Source  Geospatial  Foundation;  and  the  widespread  Google  Earth,  the  base  version  of 
which  is  currently  offered  free  by  the  Google  Corporation.  This  last  offers  very  limited  analysis 
capabilities,  but  provides  a  quick  way  to  combine  geographic  data  with  pre-supplied  aerial 
photography  and  road  network  information. 

While  different  software  uses  different  formats,  two  widespread  formats  deserve  mention  here. 
ArcGIS  stores  data  in  "shapefiles."  This  is  a  slight  misnomer,  since  a  shapefile  is  actually  a  collection 
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of  at  least  three  distinct  disc  files.  While  originally  intended  for  ArcGIS,  the  shapefile  format  is  now 
openly  available  and  in  fairly  wide  use  in  other  GIS  products.  Shapefiles  generally  hold  one  of  four 
types  of  displayable  data:  points  (such  as  locations  of  specific  incidents),  lines  (such  as  roads  or 
rivers),  polygons  (such  as  the  boundaries  of  states  or  provinces),  and  raster  data.  The  first  three  are 
collectively  known  as  "vector"  data.  Raster  data  appears  in  grid  form;  it  might  be  physical,  as  with  a 
photographic  image  of  the  ground,  or  logical,  as  in  a  map  showing  the  availability  of  fresh  water  for 
each  cell  in  a  gridded  region. 

A  second  format  is  KML,  a  text-based  format  built  in  XML  and  supported  by  Google  Earth.  KML  files 
can  contain  both  vector  and  raster  data.  It  should  also  be  noted  that  the  widely  used  open  source 
statistical  environment  R  can,  with  the  proper  libraries,  read  and  write  both  shapefiles  and  KML. 

The  construction  of  a  geographic  image  generally  consists  of  the  superposition  of  "layers,"  each 
layer  consisting  of  a  shapefile  or  KML  file  describing  some  feature  of  the  area  being  depicted. 
Because  these  tools  were  designed  for  map-makers,  the  features  are  almost  always  physical,  but  in 
principle  they  might  be  lines  showing  connections  among  individuals,  numbers  of  website  hits  per 
square  mile,  or  other  non-physical  data.  As  an  example  of  the  superimposition  of  layers,  Figure  12 
shows  three  layers  used  to  build  a  map  of  Nigeria:  a  satellite  image  (top  left),  state  boundaries  (top 
right),  terrorist  events  (bottom  left),  and  then  the  three  superimposed  (bottom  right).  Of  course,  in 
making  the  map  the  researcher  must  select  not  only  the  layers  but  their  attributes— color,  point 
size,  line  width,  transparency,  and  so  on. 


Figure  12.  Three  layers  in  a  Nigeria  map  (upper  left  and  right  and  lower  left)  and  the  resulting  map 
in  the  lower  right. 
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2.3.3.  Coloring  the  map 

The  researcher's  task,  then,  is  to  use  the  correct  set  of  layers  to  display,  and  not  conceal,  the 
essential  information.  A  map  intended  to  show  population  density  by  province  would  not  benefit 
from  the  inclusion  of  railroads  or  precipitation  unless  part  of  the  story  were  that  population  tends 
to  settle  along  the  railroads,  or  in  areas  of  higher  rainfall. 

In  the  context  of  human  activity,  data  is  often  produced  in  the  form  of  counts  by  region,  say  a 
province.  A  natural  device  is  to  color  areas  according  to  the  value  represented.  Such  a  graph  is 
sometimes  called  a  "chloropleth."  However  researchers  should  in  general  display  not  counts  but 
count  density— that  is,  counts  per  area— particularly  when  areas  are  quite  different. 

One  interesting  technique  now  available  to  analysts  is  that  of  the  "cartogram,"  in  which  a  map 
shows  regions  scaled  to  represent  the  quantity  displayed,  while  retaining  their  respective  shapes  to 
the  extent  possible.  Figure  13  shows  a  map  of  the  2008  U.S.  Presidential  election,  in  which  states 
that  awarded  their  electoral  votes  to  John  McCain  are  red,  and  those  that  awarded  their  votes  to 
Barack  Obama  are  blue  in  color  (Cole,  2012).  In  this  map,  the  ratio  of  red  area  to  blue  is  about  62%, 
yet  the  size  of  the  states  visually  misrepresents  that  68%  of  the  electoral  votes  came  from  the 
states  that  voted  for  Obama.  Cole  purposefully  included  this  map  as  a  bad  example.  In  Figure  14, 
the  data  and  color-coding  are  the  same,  but  each  state  is  represented  by  an  area  proportional  to  its 
number  of  electoral  votes  (Carter,  2008).  This  map  is  about  68%  blue.  Cartograms  can  also  be 
drawn  with  curved  boundaries. 


Figure  13.  Map  showing  the  states  whose  electoral  votes  were  awarded  to  John  McCain  (red)  and 
to  Barack  Obama  (blue),  plus  the  District  of  Columbia.  About  62%  of  the  map  is  red,  but  equal  areas 
do  not  denote  equal  densities  of  electoral  votes;  Alaska's  570,000  square  miles  of  land  are  much 
more  visible  than  Washington,  D.C.'s,  68,  though  the  two  each  contribute  three  electoral  votes. 
Adapted  with  permission  from  Cole  (2012). 
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Figure  14.  Map  showing  the  electoral  votes  awarded  to  John  McCain  (red)  and  to  Barack  Obama 
(blue)  by  the  fifty  U.S.  States  plus  the  District  of  Columbia.  In  this  map,  states  are  represented  by 
areas  proportional  to  the  number  of  electoral  college  votes  to  which  they  are  entitled.  About  68% 
of  the  map  is  blue,  because  about  68%  of  the  electoral  votes  went  to  Obama.  Adapted  with 
permission  from  Carter  (2008). 


Another  approach  to  showing  the  density  of  data  on  maps  is  the  so-called  "heat  map."  A  heat  map 
represents  values  by  colors  selected  from  a  color  ramp.  Interpolation  is  used  to  estimate  the  value 
at  points  where  no  data  is  observed;  the  coloring  is  typically  performed  without  reference  to 
political  or  other  interior  boundaries.  Figure  15  depicts  predicted  Lord's  Resistance  Army  (LRA) 
activity  in  a  given  area  based  on  previous  attack  data.  Darker  colors  (within  the  lighter  area) 
indicate  regions  of  higher  probability  of  future  attacks  and  points  show  LRA  attacks  reported  after 
the  original  prediction  was  created  and  disseminated  (McCue,  Hildebrandt,  &  Campbell,  2012). 

2.3.4.  Altitude,  time,  and  animation 

Although  shapefiles  can  hold  altitude  or  elevation  data,  other  formats  are  used  more  commonly. 
These  include  the  U.S.  Geologic  Survey's  Digital  Elevation  Model;  its  successor,  the  Spatial  Data 
Transfer  Standard;  and  the  military's  Digital  Terrain  Elevation  Data.  All  of  these  formats  were 
developed  to  support  the  display  of  actual  ground  elevations  and  do  not  seem  to  be  used  widely  to 
represent  non-geographical  data  such  as  population  densities,  although  some  exceptions  are 
mentioned  below. 
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Figure  15.  Heat  map  showing  predicted  Lord's  Resistance  Army  (LRA)  activity  in  a  given  area  based 
on  previous  attack  data.  Darker  colors  (within  the  lighter  area)  indicate  regions  of  higher 
probability  of  future  attacks  and  points  show  LRA  attacks  reported  after  the  original  prediction  was 
created  and  disseminated.  Adapted  with  permission  from  McCue  et  al.  (2012). 


It  is  also  difficult  to  display  changes  associated  with  the  passage  of  time  on  a  static  map.  Animation 
offers  a  natural  tool  here,  and  many  GIS  and  other  products  make  it  straightforward  to  create  a 
simple  animation  as  a  GIF  or  PNG  file.  These  formats  can  be  viewed  in  any  web  browser,  and  the 
resulting  animation  is  easily  understood  by  an  audience.  However,  difficulties  arise  from  the 
necessity  of  creating,  storing,  and  displaying  animations  for  all  possible  combinations  of  hardware 
and  software.  More  important,  even  today  a  large  number  of  readers  still  use  paper  to 
comprehend  and  store  visualizations. 

The  simplest  way  to  show  the  passage  of  time  is  by  showing  multiple  copies  of  the  map,  each 
constructed  using  data  for  a  different  time  period.  For  example,  Figure  16  shows  the  number  of 
nurses  per  10,000  residents  for  each  county  in  Missouri  at  six  different  time  periods  (Courtney, 
2005).  Of  course,  the  focus  on  counties  could  lead  to  the  same  misinterpretation  as  in  Figure  13: 
the  independent  city  of  St.  Louis,  which  is  very  populous  but  is  not  part  of  any  county,  is  nearly 
invisible,  and  the  density  scale  is  per  10,000  residents,  not  (as  perhaps  it  should  be)  per  square 
mile.  Still,  this  sequence  of  maps  makes  it  easy  to  track  the  status  of  specific  counties  from  year  to 
year.  The  smaller  state  map  above  each  larger  one  shows  which  counties  were  declared  to  be 
HPSAs  (Health  Professional  Shortage  Areas).  They  reveal  that  the  set  of  HPSAs  remained  mostly 
constant  between  1991  and  1999,  but  that  a  big  change  occurred  between  1999  and  2001. 
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Figure  16.  Maps  of  the  density  of  nurses,  by  county,  in  Missouri,  1991-2001.  The  insets  show  the 
set  of  counties  designated  as  Health  Professional  Shortage  Areas.  Adapted  with  permission  from 
Courtney  (2005). 


We  offer  two  more  notes  on  this  picture.  First,  Courtney  (2005)  divided  the  density  scale  of  nurses 
per  10,000  people  at  values  of  13,  44,  and  100.  Analysts  can  of  course  choose  their  own  cutoff 
points,  but  they  must  make  sure  that  (a)  the  graph  tells  the  story  they  want  to  tell;  (b)  a  different 
choice  of  cutoffs  would  not  tell  a  radically  different  story;  and  (c)  where  applicable,  the  cutoffs 
correspond  to  natural  or  important  divisions.  For  example,  in  monthly  data,  6,  12,  and  24  are 
natural  divisions,  whereas  in  income  data  natural  divisions  would  be  even  thousands  or  tens  of 
thousands  of  dollars.  Second,  the  color  choice  in  this  example  lends  itself  nicely  to  printed 
reproduction  (even  in  black-and-white  as  printed  here),  but  bright  colors  that  are  very  different  on 
a  color  computer  screen  are  often  similar  in  print.  Thus,  analysts  must  choose  colors  wisely,  using 
intensity  as  well  as  hue,  and  perhaps  with  some  sensitivity  to  the  fairly  common  red-green  form  of 
color  blindness. 

As  a  final  example,  we  present  a  map  showing  the  concentration  of  arrests  for  prostitution  in  the 
city  of  San  Francisco  in  2009.  Figure  17  (McCune,  2010)  makes  it  easy  to  see  that  arrests  in  that 
year  were  concentrated  in  two  geographic  areas.  The  author  notes  that  the  data  was  "...aggregated 
geographically  and  artistically  rendered.  This  [visualization]  is  meant  more  as  an  art  piece  than  an 
informative  visualization"  (McCune,  2010,  para  11).  While  the  tools  to  create  visualizations  like 
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these,  complete  with  lighting  and  shadow  effects,  are  apparently  not  yet  in  widespread  use,  the 
pictures  may  serve  to  indicate  current  possibilities  in  the  field  of  geographic  visualization. 


Figure  17.  Visualizations  of  the  density  of  prostitution  arrests  (vertical  direction)  in  San  Francisco  in 
2009.  Adapted  with  permission  from  McCune  (2010). 


2.4.  Visualization  of  Survey  Data 

Methods  to  visualize  survey  data  have  changed  little  in  the  past  few  decades,  and  the  standard 
techniques  of  Section  2.1  apply  whether  the  data  is  collected  for  sociocultural  analysis  or  other 
reasons.  One  challenge  emerges  because  survey  data  are  typically  discrete,  often  arising  as 
responses  to  Likert  scale  questions.  Standard  displays  of  such  data  include  pie  and  bar  charts,  even 
though  these  types  of  graphics  do  not  lend  themselves  to  much  more  than  univariate  presentation. 
These  types  of  graphs  have  little  utility  for  displaying  interactions  between  variables  or 
relationships  among  multiple  variables.  Even  with  side-by-side  and  stacked  bar  charts,  users 
generally  find  it  cognitively  difficult  to  draw  comparisons  between  subsets  of  the  data  and  thus  to 
identify  complex  sociocultural  signatures. 

The  diverging  stacked  bar  chart  developed  by  Robbins  and  Heiberger  (2011)  represents  a  recent 
advance  in  survey  data  visualization.  As  shown  in  Figure  18,  the  diverging  stacked  bar  chart  centers 
the  stacked  bars,  typically  on  the  neutral  or  central  response,  and  then  each  end  of  the  bar 
diverges  from  the  neutral.  These  types  of  plots  make  comparing  response  distributions  between 
subsets  of  the  data  relatively  easy,  clear,  and  intuitive. 
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Figure  18.  Diverging  stacked  bar  chart  for  a  five-point  Likert  scale  question.  This  example  shows  the 
results  for  the  entire  sample  (top  bar)  and  then  by  five  different  demographic  variables.  The  left  set 
of  bars  shows  the  data  in  terms  of  percentages  and  the  right  set  shows  the  counts.  Adapted  with 
permission  from  Robbins  &  Heiberger  (2011). 


A  major  impediment  to  displaying  survey  data  collected  using  complex  sampling  designs  is  the  lack 
of  software  that  would  aid  in  exploratory  data  analysis,  particularly  software  that  would  allow 
social  scientists  to  visualize  the  data  easily  (yet  correctly  and  appropriately).  Specialized  software, 
such  as  SAS,  SPSS,  Stata,  and  R,  has  these  capabilities,  but  also  requires  specialized  skills  and 
capabilities.  Rix  and  Fricker  (2012)  proposed  a  proof-of-concept  solution,  but  to  date  no  dedicated 
software  solution  specifically  facilitates  visual  EDA  of  complex  survey  data.  At  issue  is  that  complex 
sampling  designs  require  sophisticated  techniques  in  order  to  analyze  the  data  correctly.  These 
methods  must  be  an  integral  part  of  the  software,  but  they  also  must  be  relatively  transparent  to 
the  user,  while  the  software  also  must  be  designed  to  support  effective  EDA,  as  described  in 
section  1.2. 

An  open  research  question  regarding  survey  data  that  have  a  geographic  component  concerns  how 
to  map  the  data  and  simultaneously  show  the  margin  of  error  by  geographic  region.  The  difficulty 
comes  about  because  this  type  of  visualization  requires  four  dimensions  (two  for  the  map  itself,  a 
third  for  the  measure  of  interest,  and  the  fourth  for  the  margin  of  error).  Most  geographic  displays 
of  survey  data  display  only  point  estimates  by  shading  or  coloring  regions  (similar  to  Figures  13,  14, 
and  16).  The  need  to  display  the  margin  of  error  as  well  follows  if  an  EDA  system,  such  as  the  one 
described  in  the  previous  paragraph,  were  used  to  display  survey  data  geographically  and  allow  for 
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"drill  down"  into  successively  smaller  geographic  regions.  As  users  look  at  successively  smaller 
regions,  or  smaller  sample  sizes,  the  margin  of  error  increases,  and  the  display  must  give  some 
indication  of  this  increasing  uncertainty  in  the  observed  estimate. 

We  are  currently  conducting  research  into  this  issue  and  are  performing  experiments  to 
understand  which  designs  and  design  elements  work  best  at  conveying  this  type  of  information. 
The  goal  is  to  produce  a  visualization  that  appropriately  communicates  the  information  without 
much  study— one  that  is  intuitively  understandable  to  the  average  consumer  of  the  information. 
Figure  19  illustrates  one  likely  solution,  in  which  the  color  levels  reflect  the  point  estimate  values 
(as  is  the  convention  with  showing  survey  data  on  maps)  and  the  uncertainty  is  displayed  via  the 
textures.  For  this  example,  larger  hexagonal  shapes  represent  higher  uncertainty  in  the  color  value 
and  smaller  shapes  represent  lower  uncertainty  (where  the  lack  of  texture  indicates  no 
uncertainty). 


Figure  19.  Illustrative  map  that  displays  survey  results  using  color  and  uncertainty  (the  margin  of 
error)  using  texture. 


Finally,  it  is  important  to  note  that,  while  this  discussion  has  focused  on  the  visualization  of  survey 
data,  accounting  for  how  respondents  visually  interact  with  self-administered  surveys  is  critical  for 
good  data  collection.  After  all,  there  is  no  point  in  stressing  good  visualization  of  data  if  that  data 
was  collected  poorly.  Dillman,  Smythe,  and  Christian  (2009)  and  Couper  (2008)  offer  visual  design 
guidelines  for  paper-  and  web-based  surveys,  respectively. 

2.5.  Visualization  for  Linguistic  Analysis 

Linguistic  data— free-form  text  from  literature  searches,  survey  responses,  incident  reports, 
product  descriptions,  web  pages,  blog  posts,  and  a  multitude  of  other  sources— is  as  difficult  to 
analyze  as  it  is  widely  available.  Although  years  of  work  have  gone  into  methods  of  analyzing  text, 
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this  field  is  still  immature.  Unlike  structured  data,  linguistic  information  is  often  complex,  subtle,  or 
even  ambiguous,  with  interpretation  depending  on  the  reader's  advance  knowledge. 

Natural  Language  Processing  (NLP)  and  computational  linguistics  represent  the  two  major 
directions  of  linguistic  analysis  to  date.  A  third  area,  keyword  search  and  retrieval,  is  well  known  to 
users  as  the  foundation  for  web  searches.  This  technique  is,  at  least  currently,  more  mechanical 
and  less  analytical.  As  a  simple  example  that  shows  both  the  limitations  of  keyword  search  and  the 
sorts  of  difficulties  facing  researchers  using  linguistic  data,  consider  the  two  sentences  "Emperor 
Napoleon  drove  his  men  towards  Moscow  in  1812"  and  "Four  years  into  the  Peninsular  War, 
General  Bonaparte  led  the  French  army  into  Russia."  These  express  almost  exactly  the  same  idea, 
and  every  reader  knows  that  "Emperor  Napoleon"  and  "General  Bonaparte"  refer  to  the  same 
person,  but  because  the  two  sentences  use  no  words  in  common,  retrieval  techniques  will  not 
detect  their  similarity. 

NLP  is  the  study  of  extracting  meaning  from  text,  perhaps  by  identifying  parts  of  speech,  named 
persons  or  places,  or  key  phrases.  Computational  linguistics  examines  the  statistical  properties  of  a 
large  group  of  documents.  These  might  include,  for  example,  frequency  distributions  over  a  set  of 
important  words.  The  two  approaches  differ  somewhat ,  but  from  the  end  users'  perspective,  they 
both  perform  the  same  set  of  tasks  on  documents.  These  tasks  include  classification  (of  documents 
into  groups  with  known  content,  e.g.,  spam  detection),  clustering  (assignment  of  documents  into 
groups  of  similar  items),  and  sentiment  recognition  (determining  whether  a  set  of  blog  posts,  say, 
shows  a  general  approval  or  disapproval  of  a  person  or  idea). 

We  should  note  here  that  our  discussion,  like  much  research  in  language  processing,  focuses  on 
materials  in  English,  and  that  analysis  of  English-language  text  generally  involves  a  number  of  pre¬ 
processing  steps.  The  first  is  the  removal  of  "stop  words":  that  is,  words  that  do  not  necessarily 
carry  inherent  meaning,  such  as  "the,"  "of,"  and  "at."  (Of  course,  stop  words  are  sometimes 
important,  as  when  naming  the  band  The  Who  or  preserving  phrases  such  as  "To  be  or  not  to  be.") 
A  second  pre-processing  step  is  "stemming,"  which  removes  varying  word  endings  so  that,  for 
example,  "weapon,"  "weapons,"  and  "weaponry"  are  all  reduced  to  the  same  word.  In  this  step 
irregular  forms  such  as  "bought"  might  also  be  converted  to  their  regular  forms.  In  languages  with 
upper-  and  lower-case  letters  such  as  English,  words  are  normally  all  converted  to  one  case.  These 
pre-processing  steps  are  almost  always  performed  early  in  the  analysis  process  by  dedicated 
software  provided  with  suitable  word  lists.  Pre-processing  for  other  languages  may,  of  course,  be 
quite  different,  although  certain  steps— for  example,  the  removal  of  very  rare  or  very  common 
words— may  be  necessary  in  any  language. 

As  a  final  preliminary,  we  observe  that  many  of  the  techniques  for  analyzing  and  visualizing 
linguistic  data  require  measures  of  similarity  or  "distance"  between  documents  (Weiss,  Indurkhya, 
Zhang,  &  Damerau,  2005).  These  distances  must  be  computed— a  process  generally  performed 
inside  software.  For  our  purposes,  we  can  think  of  these  techniques  as  representing  each 
document  in  two-  or  three-dimensional  space.  The  analysis  then  proceeds  from  there. 

There  is  no  shortage  of  tools  for  acquiring  and  processing  text.  Krallinger  and  Valencia  (2005)  list 
several  dozen  in  the  field  of  molecular  biology  alone.  The  medical  and  biological  fields  seem 
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particularly  rich  in  research  in  this  area,  perhaps  because  of  the  large  volumes  of  literature  they 
generate.  By  contrast,  techniques  specifically  for  visualizing  text  seem  to  be  in  their  infancy. 
Instead,  visualizations  exploit  existing  approaches  for  the  tasks  at  hand;  for  example,  a  simple 
histogram  might  show  word  frequencies. 

The  system  ThemeRiver™  forms  a  sequence  of  histograms  to  depict  the  changing  distribution  of 
keywords  in  a  series  of  documents  over  time.  Figure  20  gives  one  example,  taken  from  Havre, 
Hetzler,  Whitney,  and  Nowell  (2002).  The  horizontal  axis  shows  time  from  November  1960  to  June 
1961,  and  the  vertical  scale  shows  the  number  of  occurrences  of  various  themes  in  the  speeches, 
interviews,  and  articles  of  Fidel  Castro.  (The  thematic  content  was  apparently  extracted  manually.) 
The  different  colors/shades  represent  the  different  themes,  with  each  band's  thickness 
proportional  to  the  strength  of  that  theme. 
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Figure  20.  A  "Theme  River"  showing  the  strength  of  a  set  of  thirteen  themes  in  Fidel  Castro's 
speeches,  interviews,  and  articles  from  November  1959  to  June  1961.  Adapted  with  permission 
from  Havre  et  al.  (2002). 


Perhaps  the  most  common  text  mining  task  is  clustering.  When  the  number  of  documents  is  small 
these  cases  often  include  a  tree  diagram  (or  "dendrogram").  For  example,  in  Figure  21  Fleuren  et 
al.  (2013)  used  the  publicly  available  CoPub  tool  (Fleuren  et  al.,  2011)  to  cluster  abstracts  from  the 
Medline  database  that  related  to  insulin  resistance,  together  with  the  authors'  additions  of 
categories  (large  letters,  "Blood,"  "Cancer,"  etc.).  (We  note  that  the  graph  is  difficult  to  read  even 
in  the  original,  and  that  the  numbers  relate  to  part  of  the  analysis  that  is  not  directly  relevant  here.) 
This  technique  of  hierarchical  clustering  involves  measuring  the  "distance"  from  each  document  to 
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all  the  others,  and  then  successively  joining  whatever  pair  of  documents  or  clusters  are  closest 
together. 


Figure  21.  Hierarchical  clustering  from  Medline  abstracts.  Each  document  is  provided  with  a 
"distance"  to  all  others;  then  the  clustering  joins  documents  and  clusters  according  to  distance. 
Adapted  with  permission  from  Fleuren  et  al.  (2011). 


A  second,  widely  used  form  of  clustering  is  partitioning,  of  which  the  well-known  /c-means 
approach  provides  an  example.  This,  too,  has  been  applied  to  documents  with  some  success.  One 
interesting  example  comes  from  Skupin  (2004),  where  a  corpus  of  some  2,200  conference  abstracts 
was  divided  into  25  clusters  using  k- means.  The  left  panel  of  Figure  22  shows  the  positioning  of  the 
documents  along  two  axes  (it  is  not  entirely  clear  what  those  axes  are);  the  cluster  boundaries 
would  divide  the  panel  up  into  25  pieces,  if  they  were  shown.  The  colors  in  the  original  plot,  taken 
from  standard  geographical  mapping,  show  the  density  of  documents  found  in  each  region  of  the 
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cluster  space;  thus,  many  documents  cluster  near  the  labels  "management"  and  "land"  at  the 
upper  center  of  the  picture.  (The  labels,  it  should  be  noted,  are  placed  automatically,  using 
algorithms  like  the  ones  used  in  GIS  to  prevent  overlap.)  In  the  right  panel,  the  author  has 
performed  what  he  calls  "semantic  zooming."  It  shows  a  portion  of  the  left  panel  on  a  magnified 
scale,  but  uses  100  clusters,  presumably  producing  a  finer  level  of  detail.  Analysts  can  use  this 
technique  to  interact  with  this  type  of  map  to  locate  areas— that  is,  sets  of  documents  in  the 
original  corpus— of  particular  interest.  The  commercial  product  ThemeScape  produces  maps  like 
these,  which  it  calls  "content  maps." 
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Figure  22.  K- means  cluster  results  of  2,  200  abstracts.  The  left  panel  shows  the  division  of  space 
into  25  clusters  (boundaries  not  shown),  on  top  of  which  document  densities  and  keywords  are 
shown.  The  right  panel  shows  a  sub-portion  of  the  same  region  divided  into  100  clusters.  Adapted 
with  permission  from  Skupin  (2004). 


Rather  than  cluster  documents,  it  is  possible  to  represent  them  directly— for  instance  as  points  in  a 
high-dimensional  space  whose  axes  might  represent  the  frequencies  of  certain  keywords.  As  a 
concrete  example,  suppose  we  score  each  document  based  on  the  number  of  occurrences  of  the 
words  "cancer,"  "lung,"  "emphysema,"  "bone,"  and  "fracture."  Each  document  could  be  drawn  as  a 
point  in  a  space  with  five  axes,  where  each  axis  might  measure  the  number  of  times  the 
corresponding  word  appears.  A  more  common  approach  is  to  assign  weights  to  words  in  a 
document  according  to  how  frequently  they  appear  in  that  document  and  in  the  whole  set  of 
documents.  The  statistical  techniques  of  multidimensional  scaling  provide  a  way  to  reduce  the 
dimensionality  of  a  set  of  points  onto  a  two-dimensional  plane  (or  into  three  dimensions),  while 
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preserving  the  distances  between  them  to  the  greatest  extent  possible.  Examination  of  the  two- 
dimensional  projection  can  then  yield  insight  into  the  structure  of  the  original  five  dimensions. 

Of  course,  the  dimensionality  of  the  original  data  might  be  very  much  higher  than  the  five  in  our 
example.  Figure  23  shows  an  example  from  Lopes,  Pinho,  Paulovich,  &  Minghim  (2007).  Here  the 
authors  have  represented  a  set  of  574  articles  on  three  subjects  in  a  high-dimensional  space  and 
then  projected  those  points  into  two  dimensions.  (The  labels  in  the  rectangles  can  be  ignored 
here.)  The  authors  colored  the  points  manually,  according  to  the  subject  of  the  document.  The 
number  of  mismatches  (blue  points  in  mostly  red  areas,  for  example,  which  may  not  be  visible  in 
the  black-and-white  figure  reproduction)  is  small,  suggesting  that  this  technique  could  lead  to 
useful  rules  for  detecting  content. 


/  A 


Figure  23.  Example  of  multidimensional  scaling.  Points  from  a  high-dimensional  space  built  on 
keyword  frequencies  are  projected  into  two  dimensions  in  a  specific  way,  producing  obvious 
clusters  in  content.  Adapted  with  permission  from  Lopes  et  al.  (2007). 
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One  visualization  specific  to  linguistic  data  is  the  word  cloud  (or  "wordle"),  used  to  "to  present  a 
visual  overview  of  a  collection  of  text"  (Viegas  &  Wattenberg,  2008,  p.  49).  In  particular,  words  that 
appear  more  frequently  in  the  text  appear  more  prominently  in  the  cloud;  the  cloud-building 
software  arranges  placement  and  orientation.  Figure  24  illustrates  a  word  cloud  created  from  the 
NPS  Operations  Research  Department's  application  for  the  INFORMS  Smith  Prize.  The  prize  is 
awarded  to  an  "academic  department  or  program  for  effective  and  innovative  preparation  of 
students  to  be  good  practitioners  of  operations  research,  management  science,  or  analytics" 
(INFORMS,  2013).  The  word  cloud  in  Figure  24  clearly  shows  the  story  the  department  told  in  its 
prize  application:  students  and  faculty  collaborating  on  research  and  practical  applications.  This  is 
the  point  of  the  prize  and  the  department  subsequently  won  it  using  this  application.  When 
rendered  on  some  sort  of  electronic  media,  word  clouds  can  be  made  three  dimensional,  rotating, 
and/or  interactive.  Of  course,  no  formal  analyses  are  associated  with  a  word  cloud:  the  cloud  is  the 
end  result.  Russell  (2011,  Chapter  9)  provides  additional  discussion  on  this  topic. 

FISCB  data  is  often  "streaming,"  that  is,  produced  continuously  over  time.  For  example,  social 
networks,  news  aggregators,  and  blog  feeds  can  produce  large  amounts  of  data  whose  character 
can  evolve  over  time  or  change  suddenly.  While  some  work  has  been  devoted  specifically  to  mining 
streaming  data  (for  example,  Aggarwal  &  Zhai,  2012,  Chapter  6),  this,  and  the  associated 
visualization,  remain  very  much  an  open  field. 
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Figure  24.  Illustrative  word  cloud.  Adapted  with  permission  from  INFORMS  (2013). 


2.6.  Visualization  of  Twitter  and  Other  Social  Media 

Social  media  data  can  take  many  forms,  and  thus  many  visualizations  are  possible,  including  all  of 
those  already  discussed.  Typically  the  data  is  text  based,  but  that  text  can  arise  in  a  large  number  of 
contexts:  tweets,  search  engine  queries,  blogs,  e-mail,  websites,  etc.  The  data  may  be  cross- 


204  |  Visualization  for  sociocultural  signature  detection 

sectional,  in  the  sense  that  they  are  essentially  fixed  and  change  little  over  time,  or  temporal  and 
perhaps  highly  dynamic.  As  social  media  data,  they  are  frequently  dyadic,  meaning  they  arise  as  an 
interaction  between  two  individuals,  organizations,  or  entities. 

If  the  social  media  data  do  contain  information  about  interactions,  they  can  often  be  analyzed  and 
visualized  as  a  network.  For  example,  Schroeder,  Everton  and  Shepherd  (2012)  used  network 
visualizations  to  explore  what  occurred  on  Twitter  during  the  Egyptian  protests  of  2011.  To 
understand  which  users  were  significant  conduits  of  Twitter  information,  they  analyzed  over  one 
million  tweets  about  Egypt  from  just  two  days:  January  28  and  February  4,  2011.  They  say: 

Using  these  data,  we  generated  a  user-by-user  network  [Figure  25]  where  a  direct  tie  was 
drawn  between  two  users  if  one  of  the  users  sent  a  message  to  the  other,  or  a  user  retweeted 
the  message  of  another.  In  the  case  of  the  latter,  we  drew  a  tie  from  the  author  of  the  original 
message  to  the  user  who  "retweeted"  the  message.  In  the  end,  our  user-by-user  network 
included  196,670  users  with  526,976  ties  between  them  (Schroeder,  Everton  &  Shepherd, 
2012). 

However,  as  discussed  in  Section  2.2  and  as  shown  in  Figure  25,  simply  plotting  an  entire  network 
frequently  provides  little  information.  Indeed,  Figure  25  is  really  just  another  "hairball"  network 
visualization  in  the  spirit  of  Figure  7.  Thus,  the  authors  applied  an  algorithm  developed  by  Blondel, 
Guillaume,  Lambiotte,  and  Lefebvre  (2008)  to  identify  distinct  clusters  (or  communities)  within  the 
network  based  on  the  partition  of  users  that  yields  the  highest  modularity  score.  As  shown  in 
Figure  26,  the  algorithm  located  several  distinct  clusters  in  the  data,  including  news  organization 
such  as  Al  Arabiya  and  Al-Jazeera  Arabic.  The  authors  also  found  that  a  Hosni  Mubarak  parody 
account  was  quite  central  and  speculated  that  its  tweets  may  have  been  influential  in  framing 
Egyptian  grievances  during  the  revolution  (Schroeder  et  al.,  2012). 
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Figure  25.  Network  visualization  of  tweets  during  the  Egyptian  protests  for  Twitter  data  for  January 
28  and  February  4,  2011.  Adapted  with  permission  from  Schroeder,  Everton  and  Shepherd  (2012). 
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Figure  26.  Alternate  visualization  of  the  tweet  network  after  clustering  users  via  an  algorithm 
developed  by  Blondel  et  al.  (2008).  Adapted  with  permission  from  Schroeder,  Everton  and 
Shepherd,  (2012). 


When  examining  a  corpus  of  information,  which  may  be  sets  of  documents  or  other  types  of  social 
media  information,  analysts  frequently  wish  to  identify  similar  items.  To  this  end,  Figure  27 
illustrates  a  visualization  of  some  Google+  activities  using  the  Protovis  software8  (Stanford 
Visualization  Group,  2010)  showing  the  similarity  between  pairs  of  activities  (Russell,  2011,  Chapter 
7).  In  the  figure  the  arcs  indicate  a  linkage  (similarity)  between  nodes,  which  are  the  Google+ 
activities,  and  the  nodes  are  scaled  according  to  their  degree.  The  software  sorts  the  nodes  to 
minimize  visual  clutter  and  the  diagram  clearly  shows  which  nodes  have  the  largest  number  of 
connections.  Note  that  the  diagram  is  designed  to  be  interactive;  online  the  titles  can  be  omitted 
because  they  are  displayed  when  users  mouse  over  the  associated  node.  In  addition,  clicking  on  a 
node  opens  a  new  browser  window  that  contains  the  activity  represented  by  that  node. 


8  Protovis  has  been  supplanted  by  D3.js  (Bostock,  2012),  which  provides  improved  support  for  animation  and 
interaction,  but  the  software  is  still  available.  D3.js  builds  on  many  of  the  concepts  in  Protovis. 
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Figure  27.  Visualization  of  the  similarity  of  some  Google+  activities.  Adapted  with  permission  from 
Russell  (2011). 


As  previously  discussed,  signature  detection  may  involve  determining,  deriving,  or  otherwise 
looking  for  sociocultural  signatures  or  it  may  involve  observing  particular  signatures  to  detect  if  and 
whether  they  change  over  time.  Servi  and  Elson  (2013)  have  tried  to  detect  changes  in  the  mood  of 
social  media  users.  Their  method  combines  Linguistic  Inquiry  and  Word  Count  (UWC,  2013)  with  a 
mathematical  algorithm  to  follow  trends  in  past  and  present  moods  and  detect  breakpoints  where 
those  trends  changed  abruptly.  The  results  are  then  plotted,  as  in  Figure  28,  to  show  mood  changes 
over  time.  Assuming  that  the  underlying  statistic  can  be  characterized  probabilistically,  this  type  of 
temporal  change  point  detection  can  be  further  enhanced  by  employing  methods  for  statistical 
process  control  based  on  industrial  quality  control.  For  example,  see  Figures  14-16  in  Chapter  5  of 
this  book. 
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Figure  28.  Visualization  of  Twitter  mood  change  points,  where  changes  are  evident  at  (at  least)  four 
points  during  the  displayed  time  period.  Adapted  with  permission  from  Servi  and  Elson  (2013). 


Figure  29  illustrates  another  type  of  temporal  display  that  is  interactive  when  viewed  online.  This 
particular  example  shows  a  visualization  of  events  leading  up  to  and  just  after  President  John  F. 
Kennedy's  assassination  (Huynh,  2013).  The  top  portion  of  the  plot  shows  minute-by-minute  event 
detail  from  11  am  to  1  pm  on  November  22,  1963  (note  how  the  time  scale  changes  depending  on 
the  amount  of  detail)  whereas  the  smaller  bottom  strip  puts  those  two  hours  in  the  context  of  a 
timeline  that  extends  from  October  until  about  6  pm  on  that  fateful  day.  While  not  a  plot  of  social 
media  per  se,  the  generalization  of  this  type  of  plot  to  social  media  data  is  obvious.  See,  for 
example  Russell  (2011,  Chapter  3),  where  the  timeline  is  applied  to  e-mail  data. 

Analysis  of  social  media  can  also  focus  on  social  media  metadata.  For  example,  Figure  30  is  a  word 
cloud  (similar  in  spirit  to  Figure  24  but  different  in  layout)  of  Flickr's  most  popular  photo  tags.  As 
discussed  in  Viegas  and  Wattenberg  (2008,  p.  50)  the  Flickr  "tag  cloud"  provides  "an  instant 
overview  of  the  site's  pictures." 

Donath  (2002,  p.  49)  makes  the  argument  that  "all  visualizations  will  have  some  evocative  quality. 
We  do  not  think  in  pure  abstractions;  rather,  our  thinking  is  metaphoric  and  grounded  in  the 
spatial  world."  Thus,  good  visualizations  fit  with  our  naturally  developed  intuitions.  For  example,  in 
Figure  31  Donath  (2002)  uses  a  garden  metaphor,  based  on  the  work  of  Xiong  and  Donath  (1999), 
to  visualize  participation  on  a  message  board.  In  this  "PeopleGarden"  each  individual  participant  is 
represented  by  a  flower.  The  longer  participants  have  been  involved  in  the  message  board,  the 
longer  the  flower  stem,  while  the  more  they  have  posted  to  the  message  board,  the  more  petals  on 
their  flower.  Initial  postings  are  depicted  in  a  different  color  from  replies. 
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Q  Air  Force  One  leaves  Fort  Worth  Q  Jack  Ruby  seen  with  Wanda  Walker 

Q  Man  with  rifle  spotted  on  Elm  St.  Q  Eddie  Piper  speaks  to  Oswald 

O  David  Feme  takes  mysterious  drive 
9  Oswald  near  telephone  in  T5BD 
O  Oswald  alone  in  TSBD 
O  Motorcade  leaves  Lovefield  Airport 


O  Williams  left  6th  floor 
O  Police  radio  for  ambulance 
Q  Man  with  seizure  in  front  of  TSBD 
9  Oswald  seen  having  lunch 
9  TSBD  sixth  floor  empty 
9  Man  with  rifle  seen  in  TSBD 


9  Cars  seen  cruising  motorcade  area 


•  Kennedy  shot  Q  White  male  seen  running  from  TSBD  9  Office  Tippit  ca 

O  JFK  arrives  at  Parkland  Hospital  9  Homicide  Chief  calls 
9  Witness  pinpoints  window  9  TSBD  ordered  cut  off 
9  Oswald  leaves  TSBD  and  boards  bus  9  Police  radio  assassin's  location 
9  Officers  ordered  to  north  of  Elm  St.  9  Suspect's  description  on  police  radio 
9  Oswald  gets  off  bus  and  boards  taxi 
9  Washington  D.C.'s  phone  system  breakdown 


shot 


Figure  29.  SIMILE  timeline  visualization  of  events  leading  up  to  and  just  after  President  John  F. 
Kennedy's  assassination  in  Dallas  in  1963.  Kennedy  was  shot  at  12:30  pm  and  pronounced  dead 
("t.o.d."  on  the  plot)  at  1  pm.  Adapted  with  permission  from  Huynh  (2013). 
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Figure  30.  Example  of  a  tag  cloud  visualization:  Flickr's  most  popular  tags.  Adapted  with  permission 
from  Figure  3  in  Viegas  and  Wattenberg  (2008). 
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Figure  31.  Example  of  a  PeopleGarden  visualization.  Each  flower  represents  a  participant  on  a 
message  board.  Length  of  stem  corresponds  to  longer  involvement  in  the  message  board;  the 
number  of  flower  petals  corresponds  to  the  number  of  postings.  Adapted  with  permission  from 
Donath  (2002). 


3.  Discussion  and  Conclusions 

This  chapter  has  focused  on  methods  for  visualizing  many  different  types  of  data.  Each  visualization 
method  is  appropriate  for  a  specific  type  of  data,  sometimes  in  a  particular  situation,  and  may  be 
more  or  less  useful  for  signature  detection  depending  on  how  and  when  it  is  employed.  Broadly 
speaking,  signature  detection  involves  identifying  either  differences  between  subsets  of  data,  say 
geographically  or  demographically,  or  changes  that  occur  over  time.  In  this  chapter  we  purposely 
emphasized  exploratory  data  analysis  because  it  is  impossible  (at  least  within  the  constraints  of  a 
book  chapter)  to  give  a  comprehensive  treatment  of  all  possible  strategies  for  detection 
visualization.  Furthermore,  while  good  detection  strategies  may  be  obvious  in  retrospect,  unless 
the  signature  one  is  looking  for  is  well  understood  and  well  defined,  successful  prospective 
detection  will  probably  require  the  ability  to  explore  and  search  through  data  in  multiple  ways. 

As  we  have  shown  in  this  chapter,  many  visualization  approaches  are  relevant  to  detection  of 
sociocultural  signatures.  Indeed,  given  the  limitations  of  the  chapter,  in  many  ways  we  have  only 
scratched  the  surface,  particularly  in  terms  of  the  possible  variants  of  the  visualizations  shown. 
Furthermore,  given  the  ubiquitous  availability  of  significant  computing  power  and  sophisticated 
software,  considerable  innovation  is  currently  occurring  in  visualization.  Some  of  it  results  in  eye 
candy  that  is  not  particularly  well  suited  for  good  information  and  data  communication,  while 


Visualization  for  sociocultural  signature  detection  |  211 


other  efforts  have  produced  highly  effective  communication  and  research  methods.  Separating  the 
former  from  the  latter  will  become  increasingly  important.  As  Steve  Jobs  said, 

Most  people  make  the  mistake  of  thinking  design  is  what  it  looks  like...  People  think  it's  this 
veneer  -  that  the  designers  are  handed  this  box  and  told,  'Make  it  look  good!'  That's  not  what 
we  think  design  is.  It's  not  just  what  it  looks  like  and  feels  like.  Design  is  how  it  works  (Walker, 
2003). 

Jobs's  point  applies  equally  to  the  design  of  visualization  methods  and  to  the  design  of  Apple 
products.  A  visualization  should  not  merely  look  good;  it  must  also  work  well  for  communicating 
information.  For  additional  discussions  about  information  visualization,  including  open  challenges, 
see  Chen  (2005)  and  Fayyad,  Grinstein,  and  Wierse  (2001). 

While  many  visualization  methods  are  widely  accepted  and  understood,  users  may  be  unfamiliar 
with  the  new  or  emerging  types  of  data  visualization.  Even  with  existing  methods,  users  seeking  to 
display  information  efficiently  and  effectively  must  understand  the  strengths  and  limitations  of 
each  type  of  visualization.  For  example,  in  spite  of  its  ubiquity,  the  pie  chart  is  far  less  well  suited  to 
visual  comparisons  between  groups  than  the  bar  chart  because  human  beings  are  better  at  visually 
comparing  lengths  than  areas  and  angles  (Gemignani,  2006). 

A  key  point,  as  Ben  Shneiderman  has  said,  is  that  "the  purpose  of  visualization  is  insight,  not 
pictures"  (Card,  Mackinlay,  &  Shneidermann,  1999,  p.  6).  That  is,  any  good  visualization  method 
should  lead  to  the  accurate  perception  and  comprehension  that  follow  when  the  viewer  correctly 
understands  and  interprets  the  information  encoded  in  the  visualization.  Just  because  a 
visualization  looks  fancy  or  high  tech  or  "cool"  does  not  mean  that  it  communicates  information 
accurately  or  efficiently.  Hence,  particularly  when  seeking  to  design  new  methods,  researchers 
must  carefully  evaluate  those  methods  to  ensure  they  correctly  convey  the  intended  message  to 
the  viewer,  as  well  as  to  identify  improvements  or  better  visualization  methods. 

3.1.  Research  Challenges 

Several  research  challenges  remain  in  the  field  of  visualization,  particularly  as  applied  to  detection 
of  sociocultural  signatures.  Perhaps  the  most  important  challenge  is  simply  identifying  which  types 
of  signatures  and  changes  to  signatures  are  important  to  detect.  Without  such  information, 
signature  detection  becomes  an  unstructured  and  unbounded  exercise  in  data  exploration,  with  a 
high  likelihood  that  something  "unusual"  will  be  "detected."  This  brings  us  full  circle  back  to  the 
cautions  raised  in  section  1.3  of  this  chapter. 

Quantitatively  trained  social  scientists  often  address  this  type  of  problem  by,  for  example, 
distinguishing  between  exploratory  and  confirmatory  hypothesis  testing.  In  particular,  they  employ 
confirmatory  methods  as  a  guard  against  multiple  testing  that  results  in  an  increased  likelihood  of 
Type  I  errors.  Typically  this  approach  requires  the  a  priori  specification  of  one  or  more  hypotheses 
that  are  tested  against  data  unobserved  at  the  time  of  hypothesis  specification.  Similarly,  the  a 
priori  definition  of  the  signatures  to  be  detected  helps  to  guard  against  Type  I  errors. 
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Of  course,  in  many  cases  specifying  the  signatures  is  likely  to  be  nontrivial  and  highly  context 
dependent.  However,  to  the  extent  signatures  can  be  specified,  researchers  can  apply  visualization 
methods  best  designed  or  most  likely  to  facilitate  detection.  An  example  from  the  physical  sciences 
is  the  discovery  of  Pluto  in  1930  by  Clyde  Tombaugh.  Astronomers  had  previously  predicted  that 
there  was  a  planet  out  beyond  Neptune.  Given  its  expected  location: 

Tombaugh  used  the  observatory's  13-inch  astrograph  to  take  photographs  of  the  same 
section  of  sky  several  nights  apart.  He  then  used  a  blink  comparator  to  compare  the  different 
images.  When  he  shifted  between  the  two  images,  a  moving  object,  such  as  a  planet,  would 
appear  to  jump  from  one  position  to  another,  while  the  more  distant  objects  such  as  stars 
would  appear  stationary.  Tombaugh  noticed  such  a  moving  object  in  his  search,  near  the 
place  predicted  by  Lowell,  and  subsequent  observations  showed  it  to  have  an  orbit  beyond 
that  of  Neptune  (Wikipedia,  2013). 

In  this  example,  the  type  of  available  data  (photographs)  and  the  expected  signature  (motion 
against  a  stationary  background)  drove  the  methodology  used  for  detection.9 

Other  research  challenges  may  include  legal  and  regulatory  difficulties  in  gaining  access  to  data  in 
order  to  perform  visualization;  technological  challenges  related  to  designing  and  implementing 
computer  hardware  and  software  necessary  to  display  the  data  in  a  visualization;  ethical  and 
security-related  issues  related  to  managing  and  safeguarding  the  data,  particularly  if  it  is  sensitive; 
analytical  and  algorithmic  challenges  in  developing  and  displaying  the  visualizations;  and  the 
managerial  challenges  of  effectively  assembling  and  operating  an  entire  system,  particularly  if  the 
visualizations  are  depending  on  massive  and/or  disparate  data.  Other  chapters  of  this  book  address 
some  of  these  challenges. 

Finally,  simply  continuing  to  improve  upon  existing  visualization  methods  is  an  important  research 
challenge.  Better  visualizations  require  innovation  in  the  way  data  is  visually  presented  and 
subsequent  careful  evaluation  of  those  methods  to  ensure  that  they  fulfill  their  intended  purpose. 
Whether  or  not  a  visualization  method  "works"  encompasses  a  number  of  dimensions,  including 
whether  the  visualization  presents  the  information  accurately,  whether  the  viewer  can 
subsequently  retrieve  the  desired  information  from  the  visualization,  whether  the  visualization  is 
intuitive  and  easy  to  interpret,  etc. 

Over  the  years,  researchers  have  used  formal  experiments  and/or  Darwinian  selection  to  evaluate 
many  of  the  classical  visualization  methods  noted  in  section  2.1.  Carpendale  (2008)  and  Cleveland 
&  McGill  (1984),  among  others,  offer  discussions  of  experimental  approaches  to  evaluate 
visualization  methods  and  techniques.  However,  Perer  and  Shneiderman  (2006)  propose  a  case 
study  approach  rather  than  controlled  experiments  in  a  laboratory  when  assessing  more 
complicated  visualizations,  particularly  for  evaluating  software  used  for  EDA  and  visualization. 
Often,  for  example,  effective  signature  detection  requires  more  than  simple,  static  visualization;  it 
may  require  a  software  system  with  an  interface  design  that  allows  the  user  to  easily  and 


9  Our  thanks  to  an  anonymous  reviewer  who  suggested  this  example. 
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appropriately  explore  the  data.  Important  features  include  the  ability  to  "drill  down"  into  the  data 
for  details  (for  example,  to  facilitate  easy  identification  and  analysis  of  subgroups  and  dynamic  and 
interactive  graphics),  and  to  "tour"  through  the  data,  particularly  higher  dimensional  data  (Fricker, 
2013,  p.  109).  With  such  systems  Perer  and  Shneiderman  (2009,  p.  42)  say, 

...laboratory-based  controlled  experiments  are  less  compelling  for  information  visualization 
and  visual-analytics  (VA)  research.  VA  systems  are  often  designed  for  domain  experts  who 
work  for  days  and  weeks  to  carry  out  exploratory  data  analysis  on  substantial  problems. 
These  types  of  tasks  are  nearly  impossible  to  reconstruct  in  a  controlled  experiment  for  a 
variety  of  reasons. 

Ultimately  the  point  is  that,  regardless  of  how,  new  visualization  methods  and  software  should  be 
evaluated.  As  most,  if  not  all,  of  this  work  is  now  done  via  some  type  of  computer-based  system, 
these  evaluations  may  also  extend  into  the  realm  of  human-computer  interaction  evaluations  as 
well  (see,  for  example,  de  Graaff,  2007,  and  Hunt,  2013). 

3.2.  Concluding  Thoughts 

For  social  scientists  working  with  sociocultural  data,  this  is  a  revolutionary  time  to  be  conducting 
research.  As  Michael  Macy  says,  "Human  beings  around  the  globe  are  now  communicating  with 
each  other  using  devices  that  record  those  interactions  and  have  open  access.  I  think  this  is  an 
extraordinarily  exciting  moment  in  the  behavioral  and  social  sciences"  (Miller,  2011,  p.  1814). 
Furthermore,  the  era  when  research  data  sets  were  small  enough  that  one  could  actually  look 
though  the  raw  data  and  learn  something  from  it  is  long  past.  Today  systems  capture  and  store 
data,  in  a  wide  variety  of  formats  and  types,  at  a  rate  that  makes  effective  visualization  perhaps  the 
only  way  to  comprehend  the  data.  Welcome  to  the  era  of  data  visualization! 
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1.  Introduction  and  Overview 

Sociocultural  signatures  are  unique,  identifiable  (often  because  they  are  repeated)  features  of  the 
social  and  cultural  landscape.  That  landscape  includes  observable  individual  biometrics  (i.e., 
characteristics  or  traits,  such  as  personality  profiles,  fingerprints,  and  voice  profile),  as  well  as 
individual  and  group  sentiments  and  behaviors  (Maybury,  2010)  exhibited  in  political,  economic, 
and  social  structures. 

Detection  of  sociocultural  signatures  involves  more  than  maintaining  situation  awareness  and 
identifying  facts  in  the  physical  environment.  It  requires  a  deeper  understanding  of  what  those 
signatures  mean.  Sociocultural  analysis  helps  to  unravel  the  meanings  of  these  signatures  by 
decoding  how  and  why  people  sense  the  world  as  they  do. 

As  an  example,  one  might  observe  that,  although  Brazilian  officials  were  confident  that  they  would 
be  prepared  for  the  Confederations  Cup  in  June  2013  (a  precursor  to  the  World  Cup  2014),  at  the 
time  of  writing  this  chapter,  Brazil  had  built  only  two-thirds  of  the  soccer  stadiums  needed  by 
FIFA's  (Federation  Internationale  de  Football  Association)  mandated  deadline  (Boadle  &  Downie, 
2013).  Sociocultural  analysis  might  help  to  explain  this  by  revealing  a  pattern  in  how  processes  are 
paced  in  Brazilian  society.  Researchers  might  conclude  that  it  took  Brazilian  officials  two  years  to 
determine  which  12  cities  would  host  the  games  because  of  the  government's  strong  desire  to 
generate  the  greatest  economic  benefit  for  the  country.  The  cities  selected  could  then  have  taken 
another  long  time  to  build  the  stadiums  due  to  the  same  relationship  structures  being  used  to  find 
the  building  contractors  that  would  generate  the  greatest  benefit  for  the  organizers.  Further, 
during  the  selection  process,  time  was  on  the  side  of  the  contractors  who  could  negotiate  better 
prices  as  deadlines  approached.  At  a  macro  level,  Brazilians  are  generally  more  comfortable  with 
uncertainty  than  FIFA  officials.  In  fact,  legendary  Brazilian  soccer  player  Ronaldo  was  quoted  as 
saying,  "We  leave  everything  to  the  last  minute"  (Boadle  &  Downie,  2013). 

This  example  illustrates  how  sociocultural  analysis  involves  making  sense  of  observed  behaviors  by 
accounting  for  relationship  structures  (i.e.,  negotiating  costs  and  benefits)  that  might  explain  how 
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sentiments  and  behaviors  developed  (i.e.,  "things  will  get  done"),  as  well  as  values  (i.e.,  tolerance 
for  uncertainty)  and  beliefs  (i.e.,  "do  not  do  today  what  you  can  do  tomorrow")  that  have  guided 
individuals'  behaviors  in  a  social  context.3 

In  this  chapter,  we  discuss  the  importance  of  cross-cultural  training  and  education  (CCTE)  as  a 
prerequisite  for  robust  sociocultural  analysis,  and  recommend  methodologies  for  trainers.  We  also 
offer  suggestions  to  Department  of  Defense  (DoD)  policy  makers  who  must  support  CCTE  for 
civilian  and  military  U.S.  Government  (USG)  personnel.  We  first  introduce  a  conceptual  definition  of 
culture,  as  well  as  concepts  and  dimensions  of  culture  that  trainers  must  address  in  CCTE.  Second, 
we  identify  some  gaps  in  current  analytic  practice  that  research  should  seek  to  fill.  In  the  third 
section,  we  present  state-of-the-art  findings  from  cultural  and  cross-cultural  research  that  could 
inform  CCTE  methods  and  designs,  including  best  practices  in  CCTE.  Finally,  we  put  forward  some 
recommendations  for  implementing  CCTE  more  richly  and  effectively  in  operational  environments. 


2.  What  Is  Culture? 

Culture  gives  meaning  and  distinction  to  the  existence  of  a  group.  The  term  "culture"  encompasses 
values,  norms,  behaviors,  and  beliefs  implicitly  shared  among  members  of  a  social  system  -  defined 
as  a  group  characterized  by  meaningful  interactions  among  individuals  (Schwartz,  2009).  These 
characteristics  of  a  culture  develop  and  modify  over  time,  and  are  reinforced  by  the  interactions  of 
people  within  it.  Thus,  an  inherent  component  of  understanding  culture  is  understanding  how  the 
people  within  it  interact  and  why.  For  this  reason,  intelligence  professionals  (IPs— a  broad  term 
that  includes  analysts)  must  focus  on  patterns  of  social  interactions  that  provide  evidence  of  group 
behaviors  and  how  behaviors  and  sentiments  develop.  Starting  with  dyadic  relationships,  including 
person-to-person,  person-to-group,  and  group-to-group,  IPs  can  come  to  understand  the  more 
complex  relationship  structures  that  define  cultures,  and  explain  why  dyadic  relationships  take  a 
particular  form. 

The  term  "culture"  is  multi-layered  and  can  be  applied  to  describe  different  kinds  of  social  systems, 
including  regions,  countries,  nations,  ethnic  groups,  and  families.  While  many  associate  the  concept 
with  nationality,  not  all  social  systems  correspond  to  national  borders  (McGinn,  Weaver, 
McDonald,  van  Driel,  &  Hancock,  2008).  Thus,  we  can  refer  to  peer  group  cultures,  corporate 
cultures,  and  national  cultures. 

Culture  influences  and  is  influenced  by  social,  educational,  business,  political,  economic,  linguistic, 
legal,  and  religious  systems  (Tayeb,  1994).  It  is  both  around  and  within  us,  and  individuals  have 
cultural  signatures,  but  people  are  not  embodiments  of  culture.  By  analogy,  simply  because  the 
United  States  is  considered  a  wealthy  nation  does  not  mean  that  all  people  in  the  United  States  are 
wealthy.  Similarly,  even  within  a  relatively  collectivistic  culture  different  individuals  focus  to 
different  degrees  on  their  duty  to  their  families  and  subgroups. 

Cultural  syndromes— composed  of  norms,  beliefs,  and  values— are  often  mistakenly  used  to  assert 
qualities  of  a  person.  Although  IPs  should  use  these  syndromes  to  guide  understanding  of  a 
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person's  behaviors,  they  must  also  seek  alternative  explanations  on  the  basis  of  an  evaluation  of 
multiple  cultural  factors. 

Culture  manifests  itself  not  only  in  the  perceived  physical  world,  but  also  in  how  and  why  people 
interact  in  that  world  as  observed.  Cultural  differences  become  evident  when  people  perceive  the 
same  stimuli— be  they  social  or  physical— in  different  ways  (e.g.,  interrupting  a  person  who  is 
speaking  might  mean  someone  is  respectfully  engaged  in  a  conversation  or  disrespectfully 
uninterested  in  what  another  person  wants  to  say).  Moreover,  the  consequences  of  the  perceived 
stimuli  differ  across  cultures.  For  example,  people  with  similar  professions  in  different  national 
cultures  experience  different  strains  despite  similar  work  stressors  (Glazer  &  Beehr,  2005).  In 
Glazer  and  Beehr' s  (2005)  study,  fewer  nurses  in  Hungary  intended  to  leave  their  jobs  because  of 
anxiety  than  U.S.  nurses,  even  though  the  Hungarian  nurses  had  greater  work  overload  (leading  to 
anxiety)  than  their  U.S.  counterparts.  As  this  example  illustrates,  finding  observable  factors  or 
artifacts  that  manifest  themselves  in  the  focal  environment— such  as  rate  of  turnover— is  not  the 
end  goal,  but  the  first  step  in  detecting  sociocultural  signatures.  In  understanding  the  effect  of 
culture  on  this  behavior,  one  must  recognize  how  the  behavior  relates  to  a  stimulus.  In  this  case, 
Glazer  and  Beehr  suggested  that  in  Hungary  the  relationship  structure  between  nurses  is 
communal.  Therefore,  although  the  Hungarian  nurses  had  greater  role  overload  and  higher  anxiety 
than  U.S.  nurses,  these  might  not  provide  sufficient  motive  for  the  nurses  to  leave  their  employers 
and  put  their  colleagues  in  the  precarious  position  of  enduring  even  more  overload.  In  fact,  as 
these  data  came  from  nurses  working  at  four  hospitals  in  Budapest,  and  there  were  no  differences 
on  the  main  study  variables  across  hospitals,  it  is  entirely  possible  that  moving  to  another  hospital 
would  yield  the  same  outcomes. 

Through  sociocultural  analysis,  Glazer  and  Beehr  (2005)  explained  the  processes  and  underlying 
assumptions  that  led  to  the  observed  artifacts,  such  as  identifying  the  communal  sharing 
relationship  structure  and  cultural  values  (harmony  and  conservatism;  see  Table  1  for  definitions) 
associated  with  the  detected  outcome.  Thus,  the  foundation  for  any  CCTE  activity  must  rest  on  a 
thorough  understanding  of  concepts  that  apply  throughout  a  given  culture,  which  in  turn  will  serve 
as  the  cornerstone  for  understanding  how  and  why  observed  behaviors  are  manifested. 


2.1.  Culture-General  Concepts 

Culture-general  concepts  central  to  CCTE  include  artifacts  and  practices,  norms,  beliefs,  and  values. 

Artifacts  and  Practices.  Artifacts  are  aspects  of  culture  that  are  immediately  visible;  they  include 
tangible  objects  or  observable  practices.  Practices  are  behaviors  or  patterns  of  social  interactions, 
which  in  turn  reflect  an  underlying  set  of  rules  and  understandings.  A  group's  practices  are 
informed  by  the  group's  values  and  norms.  Because  of  this,  a  practice  may  carry  different 
implications  across  groups.  For  example,  when  a  subordinate  speaks  to  a  supervisor,  it  is  common 
practice  in  U.S.  culture  that  the  two  maintain  eye  contact,  as  this  implies  attentiveness  and 
sincerity  (Hattori,  1987).  In  Japanese  culture,  however,  a  subordinate  making  significant  eye 
contact  with  a  supervisor  is  considered  disrespectful  or  immodest  (Hattori,  1987)— likely  due  to 
how  highly  Japanese  society  values  hierarchy  (Schwartz,  1999). 
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Considered  together,  artifacts  and  practices  form  "surface  culture."  After  detecting  artifacts  and 
practices,  IPs  must  consider  how  they  came  to  be,  which  addresses  "process  culture,"  and  why, 
which  addresses  "deep  culture."  Norms,  beliefs,  and  values  represent  both  process  and  deep 
cultures. 

Norms.  Norms  include  not  only  prescribed  but  also  implicit  and  informal  procedural  rules  to  guide 
and  regulate  how  people  may  behave  (Smith  &  Berg,  1987),  and  indicate  what  individuals  "ought  to 
do"  in  a  given  situation  (Fischer  et  al.,  2009;  Hitlin  &  Piliavin,  2004).  Norms  depend  on  context.  In  a 
group  context,  social  networks  both  punish  members  for  deviating  from  norms  and  reward 
members  for  upholding  them  (Cialdini  &  Trost,  1998;  Fisher  &  Ackerman,  1998).  Different  cultures 
place  varying  degrees  of  emphasis  upon  adherence  to  social  norms  in  general  and  the  punishment 
of  those  who  violate  them  (see  Triandis's,  1994,  "tightness-looseness"  cultural  dimension,  defined 
in  Table  1):  some  groups  may  allow  considerable  deviation  from  set  norms,  while  others  enforce 
them  much  more  rigidly. 

Beliefs.  A  belief  is  a  generalized  understanding  of  how  two  concepts  relate  to  one  another  (Bar-Tal, 
1990;  Bern,  1970;  Katz,  1960).  Beliefs  represent  the  fundamental  organization  of  a  worldview: 
"generalized  expectancies"  or  "premises  that  people  endorse  and  use  to  guide  their  behavior  in 
daily  living"  (Bond  et  al.,  2004,  p.  552).  The  firmness  of  a  belief  can  vary  depending  on  the  level  of 
confidence  in  the  beliefs  truthfulness.  Beliefs  are  also  highly  variable  in  terms  of  specificity  (Leung 
et  al.,  2002),  ranging  from  very  particular— referring  only  to  specific  people,  events,  situations,  or 
other  targets— to  expansive  and  abstract,  pertaining  to  a  wide  variety  of  contexts  and  targets. 

General  beliefs  guide  behaviors  across  various  situations;  they  are  what  Leung  et  al.  (2002)  refer  to 
as  social  axioms.  These  beliefs  enable  people  to  organize  their  responses  to  the  social  and  physical 
world  around  them.  For  example,  people  with  a  high  internal  locus  of  control  may  be  more  likely  to 
underestimate  hazards  in  risky  situations  (Liverant  &  Scodel,  1960)  because  they  believe  that  they 
can  control  the  situation.  Beliefs  such  as  this  also  serve  as  explanations  for  the  kinds  of  values 
people  hold  and  why  they  observe  artifacts  as  they  do.  For  example,  a  common  U.S.  belief, 
influenced  by  a  long-term  orientation  (see  Table  1),  is  that  perseverance  is  important.  This  helps  to 
explain  why  U.S.  citizens  often  continue  to  pursue  goals  even  in  the  face  of  difficult  odds.  In  the 
context  of  collection  and  analysis,  identifying  and  understanding  others'  beliefs  provides  an 
understanding  of  the  deeper  meanings  of  artifacts. 

Values.  Values  are  abstract  principles  that  people  view  as  desirable,  right,  and  good  across 
situations  (Hitlin  &  Piliavin,  2004;  Schwartz,  1999).  These  principles  motivate  and  guide  individuals, 
helping  them  to  make  sense  of  social  systems.  Values  are  embedded  in  a  group's  norms  and 
practices  (Knafo,  Roccas,  &  Sagiv,  2011).  They  facilitate  the  choices  people  make  in  evaluating 
behaviors  and  events,  and  serve  as  motivating  factors  toward  goal  fulfillment  (Schwartz,  1992, 
1999,  2009).  The  goals  served  by  values  relate  fundamentally  to  biological  and  social  survival  (in 
terms  of  interactions  and  group  welfare).  Although  cultures  may  share  the  same  goals,  the 
behaviors  exhibited  as  people  seek  to  fulfill  those  goals  might  differ  across  cultures;  because  the 
emphasis  placed  on  certain  values  differs,  behaviors  motivated  by  the  values  differ.  Thus,  at  the 
level  of  culture,  values  can  be  seen  as  solutions  to  problems  faced  by  people  in  a  given  ecological 
environment. 
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Together,  norms,  beliefs,  and  values  form  a  culture's  dominant  profile,  also  referred  to  as  a  cultural 
syndrome.  Table  1  lists  example  cultural  factors  that  contribute  to  cultural  syndromes.  Similar  to  an 
iceberg,  the  visible  portion  of  a  culture  is  small  compared  to  what  lies  hidden  beneath.  Developing 
capabilities  to  detect  hidden  factors  that  influence  the  visible  factors  is  paramount  to  sociocultural 
analysis. 


Table  1.  Cultural  elements  and  their  conceptual  definitions 


CULTURAL  VALUES 
Hofstede  (2001):  Cultural  Values 

Individualism  vs.  Collectivism 

•  Society  views  individuals'  identities  as  distinct  and  unique  to  the  social  group 
(personal  time,  freedom,  choice,  and  competition  valued)  vs.  society  views 
individuals'  identities  as  integral  and  interdependent  (group  needs  above 
personal  needs). 

Power  Distance 

•  Society  reinforces  hierarchy  as  established  by  supervisor  power  over 
subordinates.  It  describes  how  societies  cope  with  human  inequality. 

Masculinity  vs.  Femininity  (aka.  achievement  vs.  nurturing;  instrumental  vs.  expressive) 

•  Society  values  achievement  and  wealth,  resolution  of  conflict  through 
aggression,  careers  for  men  not  women,  and  "live  to  work"  vs.  society  values 
placed  on  nurturance,  environmental  welfare,  negotiation,  women's 
involvement  in  workforce  and  politics,  "work  to  live."  The  former  deals  with  the 
development  of  group  norms,  roles,  and  leadership,  whereas  the  latter  deals 
with  social  networks  and  group  decisions. 

Uncertainty  Avoidance 

•  Society  lacks  tolerance  for  uncertainty  and  reinforces  individuals'  adherence  to 
rules  and  following  prescribed  norms. 

Long-Term  vs.  Short-Term  Orientation 

•  Society  focuses  on  future  through  planning  and  saving  vs.  society  focuses  on 
immediate  action  with  little  consideration  for  implications  on  future. 


222  |  Cross-cultural  training  and  education  for  detection 


Table  1.  continued 


Schwartz  (1999):  Cultural  Values 

Autonomy 

•  Society  reinforces  individual  choices  and  opportunities  to  be  unique.  These 
societies  encourage  individual  flexibility  in  thoughts,  ideas,  emotions,  and 
feelings  and  view  the  person  as  an  autonomous,  bounded  entity,  who  can 
legitimately  change  status  quo  and  engage  in  stimulating  activities, 
o  Intellectual  Autonomy 

o  Society  places  importance  on  individuals'  independent  pursuits  of 
desired  goals  and  creative  ideas.  These  societies  view  the  person  is 
an  autonomous  decision  maker  who  engages  in  social  exchanges 
based  on  contractual  relationships.  The  person  is  encouraged  to 
value  free  thoughts  and  ideas, 
o  Affective  Autonomy 

o  Society  places  importance  on  independent  pursuit  of  positive 

affective  experiences.  These  societies  view  the  person  is  an  agent  of 
change  in  the  status  quo  and  as  desiring  arousing/stimulating 
experiences,  feelings,  and  emotions,  such  as  excitement  and  desire 
for  a  varied  and  stimulating  life. 

Egalitarianism 

•  Society  places  importance  on  equality,  opportunities  for  all,  and  providing  help 
for  the  benefit  of  the  welfare  of  others.  These  societies  encourage  people  to  be 
autonomous  decision  makers,  who  engage  in  social  exchanges  based  on 
contractual  and  cooperative  relationships  and  pursue  prosocial  activities  for  the 
welfare  of  others,  such  as  social  justice,  peaceful  world,  freedom,  honesty,  and 
equality,  but  also  accepting  one's  portion  in  life. 

Harmony 

•  Society  places  importance  on  fitting  in  with  the  environment,  avoiding  change, 
and  engaging  in  cooperative  relations.  These  societies  encourage  individuals' 
unity  with  nature,  protecting  the  environment,  and  maintaining  a  beautiful 
world. 

Conservatism 

•  Society  places  importance  on  the  status  quo,  modesty,  fulfilling  role 

expectations,  and  maintaining  homeostasis  of  the  group  or  the  traditional  order. 
In  these  societies  the  person  is  embedded  in  a  group  of  interdependent, 
mutually  obligated  others.  These  societies  emphasize  that  personal  welfare 
depends  on  the  welfare  of  the  group.  Maintaining  homeostasis  is  paramount  in 
these  cultures.  Homeostasis  between  and  among  people  is  achieved  through 
modesty,  utilitarian  exchanges,  forgiveness,  and  order. 


Hierarchy 

•  Society  emphasizes  status  differences  and  ascribed  roles  (Schwartz,  1994).  It  places 
importance  on  allocation,  coordination,  and  differentiation  of  power,  roles,  and 
resources  in  pursuit  of  wealth.  In  these  societies,  the  person  is  embedded  in  a  group 
of  interdependent,  mutually  obligated  others,  whereby  individuals  attempt  to  get 
ahead,  even  at  the  expense  of  others,  while  also  maintaining  respect  for  and 
distance  from  authority. 

Mastery 

•  Society  places  importance  on  controlling  the  social  environment  and  getting  ahead 
through  self-assertion.  In  these  societies,  the  person  is  an  agent  of  change  and 
stimulating  activity,  often  at  the  expense  of  others.  The  society  encourages  pursuit 
of  goals,  ambition  to  succeed,  and  capability  of  controlling  one's  environment, 
including  the  social  environment. 


Triandis  (1995;  2009):  Cultural  Dimensions 

Tightness  vs.  Looseness 

•  Society  adheres  to  clear,  strict  social  norms  that  are  reliably  imposed  in  socially 
accepted  "important  situations;"  deviations  are  punished  vs.  society  accepts 
unclear  norms  and  has  greater  tolerance  toward  deviations  from  norms;  sanctions 
are  not  severe  and  are  possibly  informal. 

Simple  vs.  Complex 

•  Society  is  characterized  as  being  scarce  in  resources  vs.  information  societies,  that 
have  many  human  and  material  resources. 

NORMS 

Fischer  et  al.  (2009) 

Individualism  vs.  Collectivism 

•  Society  reinforces  personal  vs.  group  needs  or  obligations  as  evident  in  socially- 
oriented  expressions  of  (vs.  values  of)  self-perceptions,  relationships  with  others, 
goal  orientation  (for  self  or  group),  and  behaviors. 

SOCIAL  AXIOMS 

Bond  et  al.  (2005) 

Societal  Cynicism 

•  Society  reinforces  a  social  system  of  generalized  hostility  toward  its  members. 

Dynamic  Externality 

•  Society  reinforces  a  combination  of  religiosity,  belief  in  fate,  and  a  focus  on  effort 
and  control. 
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3.  Science  and  Technology  Gaps  in  Cross-Cultural  Training  and  Education 

CCTE  can  strengthen  intelligence  professionals'  (IPs')  abilities  to  detect  anomalies  in  foreign  others' 
interactions  and  behaviors,  and  to  explain  the  assumptions  underlying  them  in  order  to  forecast 
their  implications  for  security  and  to  mitigate  threat.  However,  we  have  observed  that  IPs  perceive 
themselves  to  have  limited  autonomy  in  preparing  statements  that  consider  alternative 
perspectives  that  might  explain  observed  artifacts. 

Indeed,  the  most  prominent  gap  in  the  IP's  toolkit  concerns  the  ability  to  answer  "how"  and  "why" 
observed  cultural  factors  emerged.  In  many  cases,  IPs  who  must  answer  an  intelligence  question 
cannot  explain  how  and  why  the  answers  are  manifested  as  they  are.  In  some  cases,  they  may  not 
have  the  opportunity  because  the  answer  to  why  is  not  part  of  the  intelligence  question  posed.  Too 
often,  however,  IPs  explain  artifacts  on  the  basis  of  their  own  cultural  understanding  of  an 
adversary's  behaviors,  rather  than  being  able  to  present  why  the  behavior  occurs  as  it  does  from 
the  perspective  of  the  adversary.  For  example,  in  a  recent  NBC  News  report,  Windrem  (2013) 
reported  a  cyber  analyst's  observation  that  different  Chinese  groups  engaging  in  cyber-hacking 
activities  do  not  cooperate  with  each  other  because  they  do  not  share  information.  A  close 
examination  of  the  explanation  suggests  a  lack  of  cultural  understanding:  the  lack  of  sharing  is 
indicative  of  not  cooperating,  but  does  not  explain  it.  CCTE  would  help  an  IP  recognize  that  this  lack 
of  intergroup  cooperation  is  a  manifestation  of  the  cultural  values  for  mastery  and  hierarchy. 

Cultural  sensemaking  is  a  collection  of  skills  [that]  enhance  [sic.]  and/or  modify  existing 
cultural  schemas  (models)  of  the  intelligence  professional.  These  schemas  can  restrict  and 
influence  an  individual's  decision  (understanding,  assessment  and  priority)  on  what  data  to 
collect,  and  the  consequent  meaning  of  the  behavior  being  observed.  (Green  Sands  &  Haines, 
2013,  para.  2) 

This  kind  of  sensemaking  does  not  involve  applying  sophisticated  stereotypes  (Osland,  Bird,  Delano, 
&  Jacob,  2000),  but  instead  detecting  real  cultural  signatures  in  order  to  make  sense  of  the 
situation  in  which  an  interaction  occurs  in  a  particular  environment.  It  is  also  an  enabler;  it  helps  IPs 
make  sense  of  the  situation  in  which  an  interaction  takes  place,  which  is  vital  to  forecasting  and 
mitigating  real-world  concerns.  As  Flynn,  Sisco,  and  Ellis  (2013,  p.  13)  wrote,  "the  lesson  of  the  last 
decade  is  that  failing  to  understand  the  human  dimension  of  conflict  is  too  costly  in  lives, 
resources,  and  political  will. ..to  bear."  Understanding  sociocultural  factors  in  context  is  imperative 
if  violence  is  to  be  mitigated.  According  to  Flynn  et  al.  (2013),  today's  IPs  miss  the  mark  in  detecting 
factors  that  have  a  major  bearing  on  national  security.  They  argue  that  IPs  must  develop  "social 
radars"  that  provide  fine-tuned  explanations  for  observed  behaviors.  The  term  "social  radar"  refers 
to  the  detection  of  sentiments  (attitudes,  beliefs,  and  behaviors)  in  a  focal  context  (Maybury, 
2010).  Social  radar  would  enable  IPs  to  detect  when  changes  occur  in  the  social  environment  (Costa 
&  Boiney,  2012).  Costa  and  Boiney  claim  that  the  goal  of  social  radar  is  to  "take  a  data-to-decision 
support  perspective,  allowing  [IPs]  to  tailor  and  weight  the  fusion  of  indicators,  draw  on  online 
sources  to  update  model  parameters,  and  use  course  of  action  models  to  provide  quantitative 
evidence  for  indicator  integration  strategies"  (p.  5). 

Through  this  process  IPs  must  be  able  to  interpret  detected  artifacts  and  apply  their  culture 
knowledge  to  explain  how  and  why  the  artifacts  manifest.  Thus,  a  major  learning  objective  of  CCTE 


Cross-cultural  training  and  education  for  detection  |  225 


would  be  to  enable  IPs  to  develop  a  wide-angled  cultural  perspective  when  analyzing  the  actions  of 
people  who  come  from  cultures  that  differ  from  that  of  the  IP.  They  should  learn  to  consider 
multiple  alternative  explanations  for  the  observations.  Enabling  CCTE  objectives  include: 

•  Become  aware  of  one's  own  cultural  biases  and  cultural  lens,  that  is,  self-awareness 

•  Develop  skills  for  self-reflection  in  perspective  taking 

•  Develop  skills  for  detecting  and  interpreting  cultural  nuances  by  understanding  deep, 
process,  and  surface  culture 

•  Increase  awareness  of  communication  patterns  among  members  of  a  culture  (e.g.,  task 
vs.  relationship  orientation,  status,  and  roles). 

3.1.  Self-Awareness 

One  of  the  shortcomings  that  IPs  must  overcome  is  minimal  awareness  of  their  own  cultural  biases 
when  interpreting  others'  behavior.  Attribution  bias— in  which  people  explain  their  own 
experiences  as  a  function  of  the  environment,  but  others'  behaviors  as  a  function  of  their 
personality  or  personal  attributes— might  lead  IPs  to  misattribute  explanations  to  personality 
instead  of  situation  and  culture.  Thus,  the  first  step  in  enhancing  intercultural  interactions  within 
the  intelligence  community  consists  of  learning  about  or  reflecting  upon  our  own  character  and 
past  experiences. 

Self-awareness  inventories  enable  people  to  explore  their  own  thinking  patterns  and  behavioral 
styles.  These  tools  can  serve  as  a  springboard  for  thinking  about  how  those  patterns  bias  the 
interpretations  of  artifacts.  They  also  provide  guidance  for  trainers  who  need  to  understand  the 
stage  that  trainees  have  reached  in  their  development.  Objectives  associated  with  self-assessment 
are:  (a)  provide  instrumented  feedback  to  individuals  regarding  their  self-construals,  values,  and 
level  of  intercultural  competence,  (b)  introduce  training  concepts,  (c)  supply  non-threatening 
vocabulary,  (d)  serve  as  a  frame  of  reference,  (e)  teach  trainees  to  appreciate  the  strengths  and 
understand  the  limitations  of  people  different  from  themselves,  and  (f)  help  individuals  explore 
ways  to  improve  their  effectiveness  as  senders  or  receivers  of  intercultural  communication.  At  the 
end  of  a  self-awareness  training  module,  participants  should  better  understand  themselves  and 
have  a  general  sense  of  the  areas  in  which  they  must  improve  to  enhance  the  quality  of  their 
intercultural  interactions.  They  will  also  begin  to  understand  the  constraints  placed  on  them  when 
engaging  in  intercultural  interactions. 

3.2.  Perspective-Taking 

The  next  step  in  the  CCTE  process  is  learning  to  take  the  other's  perspective.  Perspective-taking 
"encompass[es]  the  ability  to  perceive  events  the  way  others  do  and  understand  how  other 
people's  cultural  values  and  assumptions  affect  their  behavior.  Perspective  taking  helps  minimize 
the  various  kinds  of  cognitive  and  cultural  bias[es]  an  intelligence  [professional]  may  encounter 
during  analysis"  (Green  Sands  &  Haines,  2013,  para.  2).  To  take  another  person's  perspective, 
people  must  relinquish  control  over  their  own  cultural  lens  and  suspend  their  own  belief  and  value 
systems  "in  order  to  experience  an  alternate  affective  or  cognitive  state"  (Abbe,  Gulick,  &  Herman, 
2007,  p.  20).  In  the  intelligence  community,  various  techniques  can  help  IPs  assume  alternative 
perspectives  and  challenge  analytic  assumptions,  including  "analysis  of  competing  hypotheses" 
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(Heuer  &  Pherson,  2011)  and  "red  teaming"  (Green  Sands  &  Haines,  2013;  Mateski,  n.d.).  The  goal 
is  not  to  provide  sophisticated  stereotypical  explanations  that  label  cultures  as  falling  within  one 
category  of  a  single  cultural  dimension  (e.g.,  either  individualistic  or  collectivistic),  but  to  untangle 
the  complex  web  of  cultural  syndromes  that  together  create  a  unique  cultural  explanation  within  a 
particular  situation  and  context  (Osland,  Bird,  DeLano,  &  Jacob,  2000). 

3.3.  Surface,  Process,  and  Deep  Culture 

Once  IPs  understand  the  complexities  of  cultural  syndromes  and  individual  profiles,  they  should 
learn  how  to  identify  artifacts  or  indicators  that  together  could  illuminate  a  cultural  signature.  For 
example,  an  IP  might  observe  a  paved  highway  that  ends  before  it  connects  to  another  roadway. 
The  IP  must  then  explore  what  the  highway  symbolizes  to  the  community  and  what  its  completion 
would  mean  to  the  people  in  the  cultural  setting.  The  IP  might  discover  that  the  community  views 
the  incomplete  road  as  a  government  ploy  to  show  citizens  that  it  is  working  toward  fulfilling  its 
promises,  but  at  the  same  time  needs  the  community  to  contribute  to  the  completion  of  the  road 
by  supporting  the  re-election  of  the  ruling  party.  Then  the  IP  can  begin  to  probe  the  layers  of 
assumptions  and  underlying  belief  structures.  Perhaps  there  is  a  generalized  pattern  of  societal 
cynicism,  exemplified  by  beliefs  that  people  "will  stop  working  hard  after  they  secure  a 
comfortable  life"  and  "powerful  people  tend  to  exploit  others"  (Leung  et  al.,  p.  293).  The 
underlying  cultural  values  that  lead  to  this  generalized  belief  include  hierarchy  and  mastery- 
principles  that  guide  status  and  rules  for  dominating  others.  The  IP  must  therefore  peel  back  the 
surface  layer  of  artifacts,  the  middle  or  process  layer  of  meaning,  and  the  deep  core  of  assumptions 
to  understand  a  culture. 

3.4.  Increasing  Awareness  of  Communication  Patterns 

When  observing  dyadic  interpersonal  communication,  IPs  must  first  have  a  good  model  of  the 
observable  elements  that  identifies  if  and  how  the  elements  are  connected:  who  communicates 
with  whom;  what,  when,  and  how  often  they  communicate;  and  what  types  of  information  are 
exchanged.  IPs  can  apply  social  network  analysis  to  gain  this  insight.  A  second  fundamental  aspect 
of  communication  concerns  the  role  that  each  individual  plays  in  the  communication  network  (e.g., 
primarily  a  sender  or  a  receiver).  Role  theory  can  help  to  illuminate  this  aspect.  Finally,  IPs  must 
consider  the  detailed  characteristics  of  the  communication  that  occurs,  including  the  semantics, 
grammar,  syntax,  and  other  nonverbal  factors  that  together  can  yield  a  cultural  signature. 
Together,  these  components  form  the  basis  for  understanding  relationship  structures. 


4.  State-of-the-Art 

This  section  describes  state-of-the-art  findings  on  perspective  taking  and  communication  patterns 
that  inform  CCTE  designs. 

Researchers  have  made  numerous  advances  in  the  past  decade  that  assist  people  to  take  on 
different  cultural  perspectives  and  improve  understanding  of  relationship  structures  (Dien,  Blok,  & 
Glazer,  2011).  These  findings  have  yet  to  be  incorporated  in  CCTE,  as  the  protocols  for  perspective¬ 
taking  and  network  analysis  for  identifying  sociocultural  signatures  in  relationships  have  not  yet 
been  tested  outside  of  the  laboratory.  Below,  we  describe  the  latest  findings  on  cultural  priming  as 


Cross-cultural  training  and  education  for  detection  |  227 


a  technique  to  support  perspective  taking,  as  well  as  network  analysis,  role  theory,  and  relationship 
structures  as  tools  for  detecting  and  explaining  identified  artifacts.  CCTE  programs  should 
incorporate  findings  from  these  research  streams  to  the  extent  they  are  relevant  to  each  agency's 
CCTE  goals. 

4.1.  Perspective-Taking 

One  of  the  enabling  objectives  of  CCTE  is  to  prepare  IPs  to  consider  alternative  perspectives. 
Current  research  from  laboratory  studies  offers  promising  options  for  CCTE.  Researchers  have 
demonstrated  methods  that  cue  people  to  view  artifacts  through  a  cultural  lens  (e.g.,  individualism 
or  collectivism)  different  from  their  own  (e.g.,  Gardner,  Gabriel,  &  Lee,  1999;  Han,  2010;  Oyserman 
&  Lee,  2008a;  Oyserman  &  Sorensen,  2009).  Cultural  priming  studies  (e.g.,  Gardner  et  al.,  1999; 
Han,  2010;  Oyserman  &  Lee,  2008b;  Oyserman,  Sorensen,  Reber,  &  Chen,  2009)  have  provided 
behavioral  evidence  through  application  of  psychological  and  neuroscientific  methodology  that 
people  from  North  America  and  East  Asia  can  be  cued  to  think  from  different  cultural  perspectives, 
as  evidenced  by  participants'  change  in  endorsed  values  (Briley  &  Wyer,  2001,  2002;  Gardner  et  al., 
1999;  Yang  &  Bond,  1980).  Several  studies  have  also  examined  the  effects  of  priming  on  judgments 
about  specific  scenarios,  such  as  acceptance  of  euthanasia  or  affirmative  action  (Kemmelmeier, 
2003;  Kemmelmeier,  Wieczorkowska,  Erb,  &  Burnstein,  2002). 

4.2.  Communication  Patterns 

Network  Analysis.  CCTE  programs  can  employ  a  number  of  tools  and  techniques  for  network 
analysis :  the  process  of  modeling  the  relationships  among  a  set  of  individuals,  groups,  or  other 
entities  based  on  data  about  them  (Heuer  &  Pherson,  2011).  Each  entity  is  a  node  in  the  network; 
analysis  identifies  and  quantifies  the  relationships,  or  linkages,  between  the  nodes.  These 
techniques  are  very  useful  for  analysis  of  dyadic  communications  in  that  each  data  point  represents 
a  connection  between  two  entities  (e.g.,  observing  the  number  of  times  Bill  initiated  a 
communication  with  George).  By  this  measure,  the  strength  of  the  link  between  Bill  and  George  is  a 
function  of  the  total  number  of  communications  exchanged  between  them.  It  is  critical,  then,  for 
IPs  to  be  aware  of  the  measurement  unit  used  to  determine  associations  (e.g.,  number  of 
communications)  when  interpreting  a  network  model. 

One  advantage  of  network  analysis  is  that  computational  algorithms  can  directly  quantify 
properties  of  the  network  as  a  whole  (e.g.,  degree  of  centralization)  or  properties  of  individual 
entities  within  the  network  (e.g.,  the  centrality  of  Bill).  Quantities  that  refer  to  regular  patterns  of 
relationships  among  the  entities  are  known  as  structural  variables  (Wasserman  &  Faust,  1994), 
because  together  they  determine  the  shape  of  the  network  at  a  given  time.  As  such,  although  these 
quantities  provide  IPs  with  the  big  picture,  it  is  also  imperative  to  distinguish  between  them  and 
the  quantities  characterizing  dyadic  relationships  between  two  entities. 

Network  analysis  also  has  the  advantage  of  allowing  IPs  to  study  the  structure  of  the  network  at 
successive  time  points  to  understand  the  development  of  an  organization  (Kossinets  &  Watts, 
2006).  IPs  can  examine  structure  at  a  number  of  levels  to  understand  interactions  among  different 
types  of  entities.  Carley  and  Reminga  (2004)  observed  that  "organizations  can  be  modeled  and 
characterized  as  a  set  of  interlocked  networks  connecting  entities  such  as  people,  knowledge, 
resources,  tasks,  and  groups"  (p.  2). 
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With  respect  to  understanding  the  culture  of  a  given  group  of  people,  a  social  network  is  perhaps 
the  primary  model  to  use  for  inferring  the  nature  of  the  relationship  between  any  two  individuals 
or  groups  and  the  function  that  relationship  serves  in  the  organization  at  large  (Borgatti,  Mehra, 
Brass,  &  Libianca,  2009;  Heuer  &  Pherson,  2011).  However,  structured  social  relationships  alone  do 
not  suffice  to  model  culture  (Carley  &  Reminga,  2004).  The  processes  by  which  resources  and 
information  are  managed  (i.e.,  distributed  and  exchanged)  among  people  living  in  a  region,  as  well 
as  the  ways  in  which  people  interact  with  objects  (e.g.,  tools,  artifacts,  icons)  are  distinguishing 
characteristics  of  culture  as  well.  By  layering  network  diagrams  of  these  and  other  such  factors 
upon  social  networks,  richer  pictures  of  the  culture  emerge,  enabling  IPs  to  study  the  effects  of 
interactions  among  these  different  modalities  on  culture. 

For  example,  IPs  can  augment  their  understanding  of  the  relationship  between  Bill  and  George  if 
their  closeness  in  the  network  reflects  not  just  the  number  of  communications  they  exchange,  but 
also  the  amount  of  time  they  are  physically  co-present  with  each  other  and  the  number  of  times 
they  exchange  money  or  goods.  If  Bill  and  George  are  known  to  communicate  and  be  co-present 
frequently,  but  have  few  exchanges  of  money  or  goods,  then  it  is  not  likely  that  the  two  men  are 
business  associates.  One  challenge  for  CCTE,  then,  is  to  improve  IPs'  abilities  to  quickly  interpret 
the  abstract  network  models  in  terms  of  the  real-world  sociocultural  signatures  associated  with  the 
modeled  data. 

Role  Theory.  One  way  for  IPs  to  interpret  sociocultural  signatures  associated  with  modeled  data  is 
to  understand  the  role  played  by  each  entity  in  the  social  structure.  Role  theory  states  that  within 
any  given  context,  each  person  has  a  role  that  is  influenced  by  expectations  (from  others)  or  the 
person's  own  interpretation  of  expectations  about  socially  acceptable  behaviors  (Beehr  &  Glazer, 
2005).  In  a  role  set  analysis,  members  of  an  individual's  role  set  include  the  focal  person,  who  is 
typically  the  role  receiver,  and  multiple  role  senders.  Role  senders  (i.e.,  people  who  have  a  stake  in 
the  focal  person's  role)  communicate  expectations  for  appropriate  behaviors  to  the  role  receiver. 
Examples  of  role  senders  include  supervisors,  peers,  subordinates,  family,  and  others  who  interact 
with  the  role  receiver.  The  role  receiver  interprets  role  senders'  communications  (or  lack  of 
communications)  and  reacts  to  the  perceived  expectation. 

As  an  example,  members  of  a  supervisor's  role  set  include  his/her  own  supervisor,  subordinates, 
peers,  family  members,  and  fellow  members  of  professional  associations.  Some  subordinates  might 
transmit  the  expectation  of  explicit  instruction;  in  response,  the  supervisor  provides  them  with 
detailed  instructions.  Other  subordinates  expect  full  autonomy,  and  the  same  supervisor  responds 
by  giving  them  only  high-level  directives  without  detailed  instructions.  If  the  relationship  between 
sender  and  receiver  is  known,  it  is  easier  to  correctly  interpret  messages  between  them  on  the 
basis  of  that  relationship. 

Relationship  Structures.  Once  IPs  have  identified  role  senders  and  receivers,  they  must  examine 
the  structure  of  the  relationship.  According  to  Fiske  (1992),  dyadic  relationships  can  be 
characterized  as  communal  sharing,  market  pricing,  equality  matching,  and/or  authority  ranking. 
Communal  sharing  refers  to  treating  individuals  within  one's  in-group  as  equivalent  to  oneself  due 
to  some  shared  similarity  or  set  of  similarities.  Individuals  in  a  community  are  all  afforded  the  same 
rights,  regardless  of  their  contributions  to  the  group.  Authority  ranking  is  defined  by  viewing 
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relationships  on  the  basis  of  position  or  status  in  a  hierarchy.  Equality  matching  refers  to 
maintaining  the  balance  of  favors  and  payments  in  a  relationship  and  either  eliminating  or  keeping 
track  of  any  imbalances  that  arise.  Market  pricing  refers  to  a  utility  and  equity-based  relationship 
that  is  determined  through  a  subjective  cost-benefit  analysis  of  proportional  contributions  to  the 
relationship. 

These  core  relational  models  (RMs)  are  considered  universal,  but  the  ways  in  which  they  are 
expressed  in  different  situations  differ  across  cultures  (Fiske,  1992).  Moreover,  an  RM  cues  a 
variety  of  detailed  prescriptions,  propositions,  precepts,  and  principles  that  might  help  explain 
individuals'  thinking  (Fiske,  2004).  A  thorough  comprehension  of  how  a  given  culture  views 
relationships  can  help  an  IP  better  understand  the  motivations  of  role  players  and  the  rationale 
behind  the  dyadic  interactions. 

Summary.  CCTE  designs  can  incorporate  each  of  the  above  approaches  to  understanding  social 
systems.  Indeed,  the  USG  already  uses  some  of  the  techniques  reviewed,  such  as  perspective 
taking  and  network  analysis,  to  improve  analytic  tradecraft.  However,  increasing  use  of  these 
techniques  does  not,  by  itself,  lead  to  increasing  cultural  awareness;  it  simply  enables  identification 
of  sociocultural  signatures.  CCTE  should  help  motivate  IPs  to  apply  these  techniques  toward 
reaching  cultural,  rather  than  merely  situational,  interpretations  of  events  or  sociocultural 
signatures.  We  therefore  propose  that  the  USG  further  improve  IP  performance  by  using  CCTE  to 
help  IPs  apply  the  techniques  to  understanding  particular  cultures. 

4.3.  Cross-Cultural  Training  and  Education 

People  in  the  United  States  tend  to  view  training  and  education  as  the  'culturally  appropriate' 
processes  for  developing  work-related  competencies,  including  competencies  for  international 
assignments  (Fowler,  1994).  Although  cultural  immersion  programs,  where  a  person  embeds  in  a 
society  for  an  extended  period  of  time,  can  also  be  helpful,  such  programs  tend  to  send  people  to 
one  country  only  and  can  be  quite  costly.  In  this  section,  we  address  the  state-of-the-art  in  CCTE, 
including  several  different  types  and  modes  of  training  that  have  proven  useful,  such  as  critical 
incidents,  role-plays,  and  scenario-based  training. 

Since  the  1990s,  much  research  has  been  devoted  to  demonstrating  the  benefits  of  CCTE,  including 
cost  reduction,  improved  performance,  and  more  effective  decision  making  (Brugman,  Reinhart, 
Feinberg,  Glazer,  Falk,  &  Castle,  2010;  Brugman,  Reinhart,  Feinberg,  Falk,  &  Castle,  2012).  CCTE 
teaches  people  to  engage  in  deeper  analysis  of  surface  observations.  Knowledge  and  skills  acquired 
from  CCTE  are  important  for  creating  cultural  fluency  and  better  understanding  of  others.  These 
competencies  give  people  tools  to  attend  to  the  big  picture,  encourage  cognitive  flexibility,  and 
reinforce  a  proclivity  for  asking  searching  questions. 

Culture-Specific  vs.  Culture-General.  Sometimes  CCTE  focuses  on  a  specific  culture  to  prepare 
people  to  work  in  or  on  a  culture  of  interest.  Although  in  the  short  term  such  training  has  benefits, 
it  does  not  produce  a  great  deal  of  transferrable  knowledge  and  or  skills  that  would  apply  to  any 
culture  of  interest.  Therefore,  the  USG  should  provide  culture-general  training  and  education  that 
would  give  students  the  declarative  and  procedural  knowledge,  skills,  abilities,  and  attitudes  they 
need  to  understand  and  work  in  or  on  different  cultures  of  interest.  For  example,  someone  who 
works  in  Russia  and  has  received  culture-specific  training  on  Russia  might  work  successful  missions 
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there,  but  might  fail  if  asked  to  work  on  Uganda  without  culture-specific  preparation.  With  culture- 
general  training  and  education,  the  learner  builds  cross-cultural  competence  and  can  more  easily 
fulfill  missions  that  call  for  shifts  in  cultural  perspectives.  Still,  both  are  needed  if  we  are  to  build 
confidence  in  our  deep  cultural  understanding. 

Training  vs.  Education.  Another  important  consideration  is  that  training  has  different  goals  than 
education.  Culture-specific  or  general  training  might  prepare  people  to  identify  artifacts  and 
determine  what  is  in  the  environment,  whereas  education  provides  tools  for  deeper  explanation  of 
how  and  why  the  observed  artifacts  were  created.  In  this  section  we  consider  different  modes  and 
media  that  the  USG  could  use  in  designing  training  or  education. 

Learning  Modes.  Didactic  learning  modes  reinforce  cognitive  development  through  passive  receipt 
of  information,  such  as  listening  to  lectures,  or  active  seeking  of  information  that  can  be  found  in 
library  resources.  Other  methods  for  imparting  information  include  briefings,  reading  materials, 
and  cultural  assimilators  (in  which  learners  are  presented  with  decision-making  scenarios  for  which 
they  are  then  asked  to  choose  a  behavioral  response  option  before  receiving  feedback  on  their 
choice).  By  contrast,  experiential  learning  is  an  active  mode  of  learning,  where  the  student 
participates  in  the  learning  process  by  practicing  behaviors  and  engaging  in  course-relevant 
content.  It  enables  students  to  develop  the  capability  of  learning  on  their  own.  Example  activities 
include  field  trips,  role-plays,  and  simulations.  See  Figure  1  for  additional  modes  of  CCTE. 
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Lectures,  briefings, films,  books, 
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incidents,  case  studies, language 
courses 


Lectures,  films,  books,  critical 
incidents,  case  studies,  self- 
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Experiential 


Role  plays,  case  studies, 
assessment  centers,  simulations, 
field  trips/studies,  visual 
imagery,  scenario-based  training 


International  potluck,  interviews 
of  international  sojourners, 
simulations,  analyzing 
international  films, 
multinational  virtual  or  joint 
learning  programs 


Figure  1.  Culture  training  and  education. 


Both  didactic  and  experiential  learning  modes  can  reinforce  students'  knowledge  and  detection 
skills  with  regard  to  each  of  the  four  enabling  objectives  (self-awareness;  perspective  taking; 
surface,  process,  and  deep  culture;  and  communication  patterns).  Experiential  activities  often 
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impart  methods  for  knowing  how,  whereas  students  learn  the  methods  for  understanding  why 
through  cognitive  learning  activities  (e.g.,  lectures  and  readings). 

Furthermore,  learning  content  for  each  of  these  modes  can  range  from  highly  specific  to  culture 
general.  An  example  of  a  culture-specific  didactic  learning  mode  might  be  reading  about  the 
culture  of  the  Druze,  whereas  a  culture-general  didactic  learning  mode  might  be  reading  about  the 
role  of  religion  in  organizing  societies. 

Either  didactic  or  experiential  cross-cultural  learning  programs  are  better  than  no  training  at  all, 
and  are  better  than  in-country  training  without  guided  learning  (Goldstein  &  Smith,  1999; 
Korhonen,  2002;  Sizoo,  Serrie,  &  Shapero,  2007;  Soeters  &  Recht,  2001),  but  neither  is  necessarily 
better  than  the  other  (Hammer  &  Martin,  1992;  Harrison,  1992).  Augmenting  experiential  CCTE 
with  didactic  CCTE  aids  learning  (Harrison,  1992;  McDaniel,  McDaniel,  &  McDaniel,  1988). 

Media  for  Training  and  Education.  Program  designers  must  also  consider  the  media  for  delivering 
CCTE.  The  three  basic  media  for  delivering  training  and  education  ar e  face-to-face  (F2F)  instruction, 
web-based  distance  learning  (DL),  and  blended  or  hybrid  learning.  The  results  of  a  controlled 
experiment  by  Gannon  and  Poon  (1997)  showed  little  difference  in  knowledge  gain  between  hybrid 
(simulation  game),  F2F  (lecture,  discussion,  group  exercise),  and  DL  (video-based  training). 
However,  MBA  students  who  took  part  in  the  hybrid  approach  felt  more  satisfied  and  considered 
the  experience  more  useful  and  relevant  than  students  who  took  part  in  the  other  learning 
experiences.  Note  that  each  of  these  media  supports  both  didactic  and  experiential  learning 
modes.  Below,  we  describe  three  commonly  used  CCTE  techniques  that  support  students' 
achievement  of  the  enabling  objectives.  Instructors  can  employ  these  techniques  with  any  teaching 
media. 

Critical  Incidents.  Critical  incidents  is  a  cognitive  training  tool  in  which  learners  must  react  to 
situations  that  mimic  real-life  critical  behaviors  and  decision  points  (Fowler  &  Mumford,  1995). 
Through  this  tool,  learners  respond  to  descriptive  situations  in  which  an  incident  requiring  cross- 
cultural  adaptation  occurred.  When  reviewing  a  critical  incident,  learners  use  their  newfound 
understanding  of  culture  and  self-awareness  to  put  themselves  in  the  place  of  the  incident's  focal 
character.  Through  self-reflection  and  discourse,  trainees  discover  their  own  implicit  biases  as  they 
are  guided  to  consider  alternative  cultural  perspectives  as  a  way  of  better  understanding  or 
appreciating  the  focal  person's  experience.  A  facilitator  knowledgeable  in  the  culture  of  interest 
encourages  the  trainees  to  discuss  what  they  learned  about  themselves  and  their  explanations  of 
others'  behaviors  and  attitudes,  how  one  personally  deals  with  a  situation  in  comparison  to  others 
in  the  training  session,  and  what  cross-cultural  skills  became  more  visibly  needed  for  the  trainee 
(Dant,  1995).  This  activity  provides  an  immediate,  instructor-facilitated  opportunity  to  utilize  new 
knowledge  and  enables  instructors  to  determine  areas  to  emphasize  further  in  training/education 
programs. 

Role-Plays.  In  role-plays,  participants  either  take  on  a  predetermined  characteristic  or  observe 
people  who  are  playing  roles.  Typically,  the  goal  for  engaging  in  role-play  is  to  develop  or  improve 
interpersonal  interactions  in  an  intercultural  setting;  the  role-plays  assist  in  skill  building 
(McCaffery,  1995).  Although  analysts  rarely  engage  in  interpersonal  interactions  in  foreign  cultures, 
role-play  can  nonetheless  help  IPs  develop  better  self-awareness  and  perspective-taking  skills, 
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improve  their  abilities  to  discovering  deep  and  process  cultural  factors  in  observed  behaviors,  and 
help  them  to  distinguish  features  of  a  particular  dyadic  communication  interaction. 

Scenario-Based  Training.  Scenario-Based  Training  (SBT)  helps  learners  develop  the  skills  needed 
for  complex  decision-making,  problem  solving,  and  adaptability.  Although  reading  about  the  results 
from  others'  active  engagement  in  an  SBT  can  be  useful  for  passive  learning,  active  participation  is 
much  more  educational.  A  well-crafted  SBT  has  a  curriculum  developed  around  a  particular 
scenario  and  provides  trainees  with  information,  demonstrations,  and  practice  in  real-world 
situations.  Facilitators  insert  events  into  the  scenario  that  form  the  basis  for  the  learning 
objectives.  The  events,  therefore,  help  students  develop  the  skills  to  think  of  alternative  ways  of 
reaching  decisions. 

This  technique  differs  from  classroom  lectures  in  that  it  uses  real-world  scenarios,  provides 
opportunities  for  practice,  and  offers  immediate  feedback  on  process  and  results.  Whereas 
listening  to  lectures  is  passive  (see  Moats,  Chermack,  &  Dooley,  2008),  participating  in  SBT  is  active 
and  therefore  has  greater  likelihood  for  transfer  of  learning  to  on-the-job  performance. 

Furthermore,  Joung,  Hesketh,  and  Neal  (2006)  found  that  training  firefighters  with  scenarios 
depicting  management  errors  and  severe  consequences  (i.e.,  error-story  training)  yields  better 
performance  outcomes  on  simulated  scenarios  than  training  that  walks  trainees  through  incidents 
managed  without  errors.  This  suggests  that  people  can  learn  from  others'  mistakes  better  than 
they  can  learn  from  others'  successes.  As  such,  SBT  can  prove  more  cost  effective,  safe,  and 
successful  at  developing  appropriate  competencies,  because  it  provides  efficient  methods  to  help 
personnel  acquire  and  transfer  complex  knowledge,  behaviors,  and  attitudes. 

4.4.  Summary 

In  this  section,  we  outlined  some  findings  from  state-of-the-art  research  on  culture,  as  well  as 
modes  and  media  for  CCTE.  An  important  point  to  make  here  is  that  we  do  not  advocate  any  one 
particular  mode  or  teaching  medium  for  CCTE,  because  training  must  be  tailored  to  the  audience 
and  need.  We  do,  however,  endorse  cross-cultural  (culture-general)  training  as  a  foundation  for  all 
DoD  and  USG  personnel  in  order  to  strengthen  cognitive  flexibility,  openness,  and  ability  to  engage 
in  sociocultural  analysis. 


5.  Transitioning  Cross-Cultural  Training  and  Education  into  Operations 

CCTE  can  contribute  to  developing  capabilities  to  discover ,  distinguish ,  and  locate  operationally 
relevant  sociocultural  signatures  derived  from  sociocultural  behavior  data.  Although  not  everyone 
has  the  opportunity  to  gain  experience  through  international  contacts,  CCTE  can  help  IPs  acquire 
knowledge  and  skills  in  detecting  cultural  nuances.  Most  CCTE  programs  focus  on  preparing 
sojourners  to  interact  with  people  in  different  cultures.  However,  in  this  type  of  CCTE,  the  goal  is 
not  necessarily  to  prepare  students  for  physical  interactions,  but  to  enable  them  to  detect  cultural 
underpinnings  in  extracted  excerpts  of  communications.  The  activities  in  which  trainees  engage 
during  CCTE  programs  can  also  be  applied  to  this  type  of  task. 

We  suggest  that  designers  of  CCTE  programs  roll  out  the  curriculum  in  stages,  corresponding  to  the 
aspects  discussed  earlier.  First,  IPs  would  receive  cultural  awareness  training  in  which  they  learn 
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about  their  own  culture  and  how  others  view  people  from  their  culture.  Once  people  become 
aware  of  their  own  cultural  orientations,  they  should  become  better  able  to  detect  differences  in 
others.  Second,  IPs  would  receive  training  in  perspective  taking  that  emphasizes  understanding 
behaviors  from  the  viewpoint  of  culturally  different  others.  Instead  of  asserting  only  a  person- 
focused  explanation,  IPs  would  evaluate  the  situation  in  which  the  behavior  occurred  and  consider 
the  degree  to  which  that  situation  might  account  for  the  behavior.  Moreover,  IPs  would  learn  not 
to  focus  too  much  on  the  most  prominent  small  elements,  or  on  only  one  area  for  discovery, 
because  doing  so  might  lead  them  to  ignore  opportunities  for  detecting  cultural  nuances.  Third,  IPs 
should  practice  their  skills  in  evaluating  surface,  process,  and  deep  culture,  possibly  via  critical 
incidents  and  SBT  techniques.  Finally,  IPs  need  to  develop  their  skills  at  detecting  relevant  dyadic 
relationships  and  explaining  how  these  relationships  influence  the  observed  artifacts  and  why 
those  relationships  evolved  as  they  have.  To  that  end,  IPs  would  benefit  from  cognitive  skills 
training  that  would  improve  general  detection  skills  (e.g.,  attention,  working  memory,  critical  and 
divergent  thinking). 

5.1.  Recommendations  for  Transitioning  CCTE  Activities  to  Operations 

Scenario-Based  Training.  SBT  appears  to  be  one  of  the  most  cost-effective  ways  of  implementing 
CCTE,  but  multiple  factors  influence  the  success  of  a  particular  SBT.  These  factors  include  the 
expertise  of  the  trainer,  the  organization's  reward  systems,  the  climate  for  error  management,  the 
learning  environment,  task  requirements,  the  student's  ability  to  transfer  learning  to  real  life, 
individual  motivation,  formative  and  summative  evaluation  strategies,  content  of  the  training 
program  (including  method,  strategy,  and  tools),  student's  real-work  life  experience  (i.e., 
accumulated  time  on  the  job),  and  the  student's  intellect  (Salas,  Priest,  Wilson,  &  Burke,  2006). 

Developing  an  SBT  is  time  consuming  and  costly  upfront,  but  in  the  long  run  the  benefits  outweigh 
the  costs,  as  the  program  can  be  used  with  all  trainees.  The  upfront  costs  cover  selecting  and 
populating  the  relevant  scenarios  (e.g.,  pulling  archival  intelligence  that  might  be  relevant  to  the 
Boston  Marathon  bombers)  from  which  designers  can  identify  the  competencies  necessary  for 
students  to  perform  effectively  on  the  job,  as  well  as  any  deficiencies  in  performance  that  require 
correction.  If  the  scenario  targets  the  right  deficiencies,  designing  and  implementing  SBTs  that 
focus  on  the  upgraded  competencies  becomes  easier,  as  does  evaluating  the  training's  success. 
Furthermore,  once  core  competencies  have  been  established,  trainers  can  generate  corpora  of 
scenarios  and  assessment  instruments  for  each,  and  then  store,  modify,  update,  and  reuse  them. 
This  would  promote  comparability  of  performance  over  time  across  agencies  and  training  sessions. 

Social  Relational  Mapping.  We  also  suggest  a  new  approach  to  training  that  draws  from  the  theory 
of  communication  patterns  (social  network  analysis,  role  theory,  and  relational  models  theory).  The 
social  relational  mapping  method  borrows  from  the  methodology  of  social  network  analysis,  in 
which  nodes  and  linkages  between  nodes  are  represented,  as  well  as  Relational  Models  Theory, 
which  states  that  any  two  individuals  or  entities  interact  in  ways  that  support  patterned  social 
behaviors.  Learners  in  this  type  of  training  would  become  skilled  at  asking  and  answering  deeper 
questions  about  the  relationship  dynamics  (structural  and  functional)  underlying  the  situations 
under  consideration.  For  example,  is  the  relationship  hierarchical,  egalitarian,  self-serving,  or  other- 
serving?  Do  people  see  others  as  they  see  themselves  (communal  sharing)?  Are  there  clear 
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demarcations  between  superiors  and  subordinates  (authority  ranking)?  Are  people  in  the  social 
network  studied  of  equal  status  to  the  focal  person  (equality  matching)  or  are  they  viewed  as  tools 
for  getting  what  the  focal  person  needs  (market  pricing)? 

As  an  example,  IPs  might  detect  that  women  in  Pakistan  have  the  legal  right  to  vote,  but  in  practice 
cannot  go  out  to  vote.  This  might  appear  to  be  a  clear  instance  of  equality  matching  vs.  authority 
ranking,  but  the  explanation  for  this  violation  of  equality  matching  and  adherence  to  authority 
ranking  might  lie  in  the  value  of  honor.  Therefore,  we  recommend  documenting  the  described 
relationships  between  nodes  (or  role  players)  in  order  to  strengthen  the  construction  of  the 
sociocultural  context  in  which  any  given  focal  person  is  embedded.  In  so  doing,  IPs  will  become 
better  able  to  (1)  take  on  the  perspective  of  the  focal  person,  (2)  explain  why  links  with  other 
nodes  are  not  as  salient  or  strong,  and  (3)  consider  ways  to  protect  and  defend  within  the 
constructed  sociocultural  context. 

5.2.  CCTE  Learning  Sequence 

Drawing  upon  practices  that  trainers  use  with  sojourners,  such  as  training  pre-departure  and  post¬ 
arrival,  we  also  recommend  that  operational  organizations  use  training  as  an  introduction,  and  also 
provide  continuous  learning  and  reinforcement  to  sustain  IPs'  abilities  to  adopt  different  cultural 
perspectives.  Ideally,  this  would  occur  by  default  in  immersion  experiences,  but  our 
recommendations  apply  in  the  cases  where  those  opportunities  are  not  available  or  cost  effective. 

As  with  pre-departure  training,  a  goal  is  to  establish  learners'  expectations  about  working  with 
cultural  materials.  Such  training  would  reduce  students'  anxieties,  increase  their  confidence  that 
they  can  detect  relevant  cultural  information,  and  reassure  them  that  all  information  observed  is 
culture  laden.  Similar  to  post-arrival  training,  on-the-job  reinforcement  of  culture  studies  would 
address  real-time  issues  as  experienced  in  the  actual  work  setting.  This  is  all  the  more  important  as 
IPs  can  become  so  immersed  in  their  subject  matter  that  they  are  sometimes  unable  to  distinguish 
between  their  own  cultural  biases  and  those  of  the  culture  they  are  analyzing. 

For  this  reason,  we  recommend  a  four-pronged  approach  to  learning:  listening,  practicing, 
reviewing,  and  engaging.  Listening  refers  to  all  the  upfront  learning,  as  well  as  ongoing  learning 
that  results  from  interacting  with  fellow  IPs.  Practicing  also  occurs  during  training,  but  in  addition 
can  occur  through  red  teaming  with  others  who  share  similar  cultural  cases.  Reviewing  includes 
reading  layperson's  materials  (e.g.,  tour  books,  books  about  a  society  that  can  be  found  in  the 
library  or  bookstore,  and  news  journal  articles),  asking  questions  of  cultural  experts,  and  reminding 
oneself  of  one's  own  cultural  biases.  Finally,  engaging  refers  to  the  habitual  practice  of  preparing 
commentary  and  collateral  evidence  on  all  analytic  tasks.  It  addresses  the  need  to  obtain  critical 
feedback  from  other  experts  on  a  focal  cultural  group,  as  well  as  from  people  who  are  critical 
evaluators  of  detecting  cultural  nuances.  As  in  software  engineering,  a  checks  and  balances  (or 
Quality  Assurance)  process  must  confirm  that  analytic  products  are  appropriately  framed  in  a 
sociocultural  context. 

Finally,  it  must  be  noted  that  one  of  the  challenges  in  transitioning  CCTE  into  operations  is  ensuring 
that  those  who  conduct  it  are  themselves  experts.  Trainers  must  have  deep  intercultural 
experience,  a  strong  foundation  in  social  psychological  processes,  and  the  ability  to  see  big  picture 
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influences.  Moreover,  IPs  should  work  with  people  who  are  experienced  with  a  cultural  target  to 
obtain  guided  critical  feedback  in  their  sociocultural  analysis. 

5.3.  Evaluating  Operational  Utility 

Empirical  inquiry  has  just  begun  to  explore  the  extent  to  which  cross-cultural  training  has  proven 
useful  for  IPs  or  military  personnel  deployed  abroad.  Criteria  for  performance  measurement  are 
still  in  development,  and  may  not  be  available  for  a  few  more  years.  Even  so,  organizations  have 
ways  to  determine  the  utility  of  CCTE  programs.  One  indicator  of  CCTE  success  would  be  a 
noticeable  increase  in  the  depth  or  richness  with  which  IPs  interpret  a  situation.  This  measure 
would  require  a  comparison  of  baseline  analytic  performance  to  performance  after  CCTE. 
Instructors  might  evaluate  the  utility  of  CCTE  programs  across  multidisciplinary  and  multi-agency 
teams  of  IPs,  including  technology  experts  and  social  scientists. 

Furthermore,  instructors  can  develop  evaluation  forms  for  subject  matter  experts  to  use  in 
evaluating  IPs'  reports  prior  to  training  and  then  again  after  have  had  a  chance  to  revise  their 
reports  post-training.  Evaluators  would  look  for  content  changes  that  relate  to  cultural 
understanding,  such  as  those  found  in  collateral  or  commentary,  indicative  of  more  expert 
understanding  of  the  culture.  We  would  expect  that  IPs  who  receive  cross-cultural  training  would 
provide  more  culturally  informed  and  relevant  explanations  in  their  products  than  IPs  who  had  not 
received  the  training. 

Another  measure  of  CCTE  effectiveness  is  whether  analytic  reports  more  accurately  characterize, 
explain,  or  forecast  future  events  in  a  given  situation.  This  is  a  difficult  measure  to  derive,  however, 
given  that  no  one  can  be  certain  that  sociocultural  analysis  was  accurate  until  an  event  occurs  and 
the  analysis  is  found  to  have  flaws.  Airlines  have  learned  to  prevent  accidents  and  incidents  by 
studying  data  available  from  FAA  logs,  and  therefore  now  take  preparatory  steps  to  enable  flight 
crews  to  communicate  with  each  other  and  tower  control  through  training  and  education 
(McIntyre,  2000).  Similarly,  scholars  of  national  culture  must  study  factors  omitted  from  earlier 
reports  that,  viewed  through  a  sharp  cultural  lens,  would  have  indicated  cultural  signatures  alerting 
to  imminent  danger.  Through  this  type  of  evaluation,  CCTE  designers  can  ensure  inclusion  of 
relevant  cultural  factors  that  would  become  required  features  of  improved  CCTE  for  IPs. 

Ultimately,  the  return  on  investing  in  CCTE  will  take  the  form  of  increased  security  from  foreign 
threats  and  smoother  interactions  with  foreign  national  diplomats,  but  transitioning  CCTE  into 
operations  requires  policy  makers  to  reinforce  policy  directives,  such  as  Intelligence  Community 
Directive  2.0.3,  Analytic  Standards.  This  directive  encourages  inclusion  of  collateral  and 
commentary  in  reports. 

We  also  strongly  recommend  that  members  of  the  analytic  community  work  with  people  outside 
their  own  groups  or  agencies.  The  sharing  of  expertise  between  people  from  different  groups  and 
agencies  can  lead  to  insights  and  perspectives  across  disciplines— a  process  very  much  akin  to 
detecting  cultural  signatures  in  the  world  at  large. 
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6.  Conclusion 

In  this  chapter,  we  argued  that  the  ability  to  detect  those  aspects  of  a  situation  central  to  defining 
the  sociocultural  context  depends  on  the  observer's  deep  knowledge  of  the  cultures  involved  and 
what  the  artifacts  mean  in  each  of  them.  We  proposed  that  an  important  step  toward  achieving 
this  knowledge  is  for  IPs  to  obtain  CCTE  that,  over  time,  will  help  them  become  more  sensitive  to 
the  relevant  sociocultural  signatures. 

As  we  have  shown,  the  key  questions  that  must  be  answered  in  order  to  detect  cultural  patterns 
are  "how?"  and  "why?"— not  just  "what?"  CCTE  enables  IPs  to  develop  awareness  of  the  ways  in 
which  their  own  culture  influences  their  thinking,  become  more  adept  at  seeing  events  or  artifacts 
from  others'  point  of  view,  and  more  clearly  understanding  the  multiple  layers  comprising  every 
culture.  We  have  also  pointed  out  how  network  analysis  and  social  radar  can  be  useful  tools  for 
mapping  potential  factors  that  contribute  to  a  social  situation,  and  that  training  in  social  relational 
mapping  can  equip  IPs  to  explain  the  observations  made  through  these  tools.  Further,  we 
recommended  considering  SBT  as  an  especially  cost-effective  approach  to  improving  people's 
sensitivity  to  culture  cues.  Given  the  findings  of  CCTE  researchers,  we  appear  to  have  good  grounds 
for  expecting  that  increased  cross-cultural  experience  will  help  IPs  provide  contextual  background 
that  policy  makers  could  use  to  forecast  and  mitigate  potential  threats  and  to  facilitate  peaceful 
operations. 
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Section  Three:  Forecasting 

Forecasting  the  sociocultural  environment  and  behaviors 

Jennifer  Mothieu  &  Les  Servi,  The  MITRE  Corporation 


Capabilities  are  needed  for  tracking  and  forecasting  change  in  entities  and  phenomena  of 
interest  along  multiple  dimensions  (time,  space,  social  networks,  types  of  behavior,  etc.) 
through  persistent  sensing  and  modeling  of  the  environment.  (Schmorrow,  2011,  p.  43) 

Flynn,  Sisco  and  Ellis  (2012)  wrote  "With  a  deeper  understanding  about  populations,  <we>  will  be 
able  ...  to  more  accurately  analyze  how  contemporary  threats  will  likely  impact  populations  and 
identify  means  for  counteracting  them  when  they  are  potentially  harmful.  But  the  process  begins 
with  a  robust  ...  capability  before  threats  manifest  themselves."  This  encapsulates  a  major 
challenge  of  HSCB  forecasting.  The  chapters  in  this  section  describe  the  state-of-the-art  capabilities 
available  to  meet  some  of  these  challenges,  as  well  as  emerging  capabilities  that  are  likely  to  be 
useful  in  the  near  future. 


Data  Processing 

The  chapter  on  data  processing  addresses  issues  related  to  representing  a  particular  sociocultural 
system  in  a  way  that  involves  only  data  about  an  actual,  ongoing  situation  that  can  be  used  to  (1) 
reveal  meaningful  changes  in  the  system  over  time  and  (2)  inform  models  that  can  help  users  to 
anticipate  those  changes.  These  characteristics  distinguish  data  processing  issues  for  forecasting 
from  those  in  the  Understand  capability  area,  in  that  both  current  and  historical  data  can  aid  in 
forecasting  sociocultural  behavior.  They  also  separate  forecasting  from  the  Detect  capability, 
where  it  suffices  for  algorithms  to  find  patterns  in  data  without  predicting  future  behavior.  The 
chapter  centers  on  selected  aspects  of  data  processing  relevant  to  dynamics-based  models  and 
illustrates  the  discussion  with  applications  relevant  to  the  dynamics  of  political  and  insurgent 
networks. 
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Computational  Modeling 

The  computational  modeling  chapter  emphasizes  that  the  nature  and  basis  of  models  used  across 
the  various  social  science  disciplines  differ,  and  also  differ  from  those  of  the  physical  models  that 
underlie  traditional  military  simulation.  Computational  sociocultural  models  require  the  ability  to 
represent  and  manipulate  imprecise  perceptual  and  cognitive  concepts.  This  chapter  describes  how 
computational  sociocultural  models  can  contribute  to  forecasting  human  behavior  and 
sociocultural  interactions,  and  account  for  natural  vagueness  of  human  thought  and 
communication,  along  with  the  dynamics,  complexity,  uncertainty,  variability  and  unpredictability 
of  actions  and  behavior.  It  discusses  11  different  types  of  computational  sociocultural  models 
chosen  to  cover  a  broad  set  of  approaches,  but  those  types  by  no  means  exhaust  the  options 
available.  (Note  that  all  the  other  chapters  in  this  section  also  discuss  dynamic  models  in  their 
respective  contexts.) 


Visualization 

The  chapter  on  visualization  examines  forecasting,  analysis,  and  visualization  using  the  'social 
radar'  metaphor  and  standard  questions  of  interest  to  analysts  of  complex  systems.  The  author 
illustrates  the  concept  by  the  example  of  the  Virtual  Strategic  and  Forecasting  Tool  (V-SAFT) 
developed  by  Lustick  Consulting,  which  represents  a  stand-alone  offshoot  of  work  performed  by 
the  Worldwide  Integrated  Crisis  Early  Warning  System  team  at  Lockheed  Martin  Advanced 
Technology  Laboratories.  The  discussion  uses  the  state  space  of  the  future  produced  by  V-SAFT  as  a 
target  for  queries  designed  to  distinguish  plausible  and  probable  futures  from  those  that  are 
merely  possible.  It  also  covers  how  the  tool  can  monitor  change  across  regularly  updated  versions 
of  that  state  space  in  order  to  generate  indications  and  warnings,  and  can  offer  opportunities  for 
what-if  experiments  to  clarify  the  implications  of  different  assessments  of  key  variables  in  the  real 
world. 


Training 

The  training  chapter  describes  the  current  state  of  the  science  and  technology  related  to  training  in 
sociocultural  forecasting,  and  identifies  gaps  that  research  must  address  to  develop  capabilities  for 
application  in  operational  settings.  The  chapter  begins  with  an  overview  of  forecasting  in  the 
sociocultural  domain,  with  an  emphasis  on  forecasting  change  in  cultural  behavior  and  social 
structure.  It  explores  the  relationship  between  sociocultural  sensemaking  and  forecasting  and 
presents  a  use  case  to  illustrate  the  problem.  The  chapter  then  surveys  the  literature  on  training 
for  forecasting,  with  an  emphasis  on  probabilistic  forecasting,  and  highlights  special  problems 
associated  with  forecasting  in  the  sociocultural  domain,  along  with  their  training  implications. 
These  include  the  nature  and  range  of  problems  being  predicted,  from  short-term  political  actions 
and  events  to  long-term  change  in  a  population's  cultural  values,  beliefs,  and  behaviors. 


Section  introduction:  Forecasting  |  243 


References 

Flynn,  M.  T.,  Sisco,  J.,  &  Ellis,  D.  C.  (2012)  Left  of  bang:  The  value  of  sociocultural  analysis  in  today's  environment.  PRISM , 
3(4),  12-21. 

Schmorrow,  D.  (2011).  Sociocultural  behavior  research  and  engineering  in  the  Department  of  Defense  context.  Washington, 
DC:  Office  of  the  Secretary  of  Defense,  Assistant  Secretary  of  Defense  for  Research  and  Engineering,  Human 
Performance,  Training,  and  BioSystems  Directorate. 


Data  processing  for  applications  of  dynamics-based  models  to  forecasting  |  245 


9  Data  processing  for  applications  of  dynamics-based 
models  to  forecasting1 

Michael  Gabbay,  University  of  Washington 


1.  Introduction 

The  Forecast  operational  capability  tracks  and  projects  change  along  multiple  dimensions  in 
sociocultural  entities  and  phenomena  of  interest.  This  chapter  addresses  issues  related  to 
representing  a  particular  sociocultural  system  in  a  way  that  involves  (a)  only  data  available  for  an 
actual,  ongoing  situation  that  can  be  used  to  (b)  reveal  meaningful  changes  in  the  system  over  time 
and  (c)  implement  models  that  can  help  users  to  anticipate  those  changes.  These  three  elements 
distinguish  data  processing  issues  for  forecasting  from  those  in  the  Understand  capability  area,  in 
which  historical  -  not  only  current  -  data  can  aid  in  understanding  sociocultural  systems.  They  also 
separate  forecasting  from  the  "Detect"  area,  in  which  it  suffices  for  algorithms  to  find  patterns  in 
data  without  predicting  future  behavior  or  supporting  causal  models  of  how  the  system  evolves. 
This  chapter  centers  on  selected  aspects  of  data  processing  relevant  to  dynamics-based  models, 
i.e.,  models  that  posit  causal  mechanisms  of  system  evolution,  rather  than  purely  statistical 
models,  and  illustrates  the  discussion  with  applications  in  the  domain  of  political  and  insurgent 
network  dynamics. 

The  first  section  of  this  chapter  briefly  describes  dynamics-based  modeling  methodologies  and 
presents  a  model  of  group  decision  making  as  an  example  that  will  help  guide  the  subsequent 
discussion.  Dynamics-based  models  involve  an  important  distinction  between  variables  and 
parameters  and  the  second  section  of  the  chapter  discusses  the  requirements  and  considerations 
involved  in  calculating  their  values.  Calculating  parameters  is  particularly  difficult  and  the  extent  to 
which  they  can  be  determined  influences  whether  the  output  of  a  model  can  be  evaluated  with 
respect  to  an  actual  time  scale  or  only  with  respect  to  equilibrium  outcomes. 

The  chapter  then  presents  two  examples  of  applications,  both  concerning  leadership  elites  and 
their  organizations.  Research  has  largely  overlooked  the  behavior  of  leaders  and  organizations 
amidst  the  surge  of  interest  in  social  media,  population  sentiment,  and  big  data,  yet  this  type  of 
data  is  particularly  important  in  tracking  and  anticipating  change  on  time  scales  of  operational 
concern  with  regard  to  entities  subject  to  operationally  relevant  levers  of  influence.  These  two 
examples,  presented  in  the  fourth  and  fifth  sections,  use  different  types  of  input  data.  The  first 
involves  the  use  of  expert  judgments  in  conjunction  with  the  group  decision-making  model.  The 
second  involves  the  representation  and  modeling  of  insurgent  network  behavior  and  shows  how 
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the  ideologies,  policies,  and  relationships  among  insurgent  groups  can  be  constructed  using 
rhetoric  as  data  and  tracked  over  time.  For  the  insurgent  network  application,  the  chapter  presents 
a  stochastic  model  of  insurgent  cooperation  at  the  tactical  level  and  compares  the  model  results 
with  the  network's  evolution  as  observed  in  the  data.  Finally,  the  chapter  provides  a  brief 
discussion  of  gaps  in  knowledge  and  future  research  directions  related  to  data  processing  and 
forecasting,  focusing  on  areas  relevant  to  dynamics-based  models  and  political  application 
domains. 


2.  Dynamics-Based  Models 

In  dynamics-based  models,  variables  can  evolve  based  on  their  present  state  and  their  interactions 
with  other  variables.  Thus,  a  variable  can  be  affected  by  self-feedback,  as  in  an  exponential  growth 
or  decay  situation  in  which  the  rate  of  change  of  the  variable  is  proportional  to  its  value,  and  by 
feedback  from  other  variables  in  the  system,  as  in  a  predator-prey  model  where  the  rate  at  which 
the  number  of  prey  animals  declines  depends  on  the  number  of  predators.  Typically,  these  models 
posit  causal  mechanisms  that  underlie  the  forms  of  the  feedbacks  included.  For  example,  in  a 
model  that  treats  the  spread  of  a  revolutionary  ideology  like  the  spread  of  an  epidemic,  the  rate  at 
which  the  number  of  people  "infected"  with  the  ideology  increases  is  taken  to  be  the  product  of 
the  present  number  of  already  infected  people  and  the  number  of  people  who  are  susceptible  but 
not  yet  infected  (Epstein,  1997).  This  assumes  that  each  interaction  between  an  infected  and 
susceptible  person  has  a  certain  chance  of  resulting  in  a  new  adherent  to  the  ideology,  as  in 
person-to-person  disease  transmission. 

This  product  interaction  between  variables  is  an  example  of  a  nonlinear  interaction.  Nonlinear 
models  can  accommodate  complex  behaviors  such  as  abrupt,  discontinuous  changes  in  the  nature 
of  the  system's  behavior.  Such  sharp  transitions  are  often  of  highest  importance  in  sociocultural 
forecasting,  e.g.,  will  a  localized  uprising  result  in  a  mass  revolution?  Indeed,  the  epidemic  model 
contains  a  critical  threshold,  called  the  reproduction  rate,  above  which  the  epidemic  or  revolution 
will  take  off  but  below  which  it  will  fizzle  out.  Purely  statistically-based  approaches  such  as 
regressions  are  ill  suited  to  forecasting  sudden  transitions  in  sociocultural  contexts,  which  is  one 
reason  why  this  chapter  focuses  on  dynamics-based  models.  In  addition,  statistically  based  models 
often  provide  little  insight  into  how  interventions  into  a  situation  will  play  out;  therefore,  they 
cannot  be  applied  to  analyzing  courses  of  action  for  the  purposes  of  the  Mitigate  capability  area. 
However,  regardless  of  whether  one  uses  dynamics-based  models  or  statistical  methods, 
sociocultural  forecasting  remains  a  difficult  task,  largely  due  to  the  lack  of  appropriate  empirical 
data. 

There  are  various  ways  of  implementing  dynamics-based  models,  three  of  which  are  briefly 
mentioned  here:  differential  equations,  stochastic  models,  and  agent-based  models  (for  a  more 
detailed  description  of  various  modeling  methods,  see  chapter  10  on  computational  models  in  the 
Forecasting  section  of  this  book).  Differential  equations  give  mathematical  expressions  for  the 
rates  of  change  in  a  system  of  variables  that  can  then  be  solved,  computationally  or  analytically,  to 
show  the  values  of  the  variables  over  time.  The  equations  evolve  variables  in  continuous  time 
(difference  equations  are  the  discrete-time  analog).  The  epidemic  model  noted  above  is  most 
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readily  expressed  and  analyzed  in  differential  equation  form.  The  Lanchester  model  for  combat 
dynamics  is  another  differential  equation  that  may  be  familiar  to  some  readers  due  to  its 
prevalence  in  the  combat  simulations  used  in  operations  research  (Epstein,  1997).  Differential 
equations  are  typically  deterministic  in  that  the  future  evolution  of  the  variables  is  precisely 
determined  by  their  initial  values  (although  they  can  be  adapted  to  accommodate  stochastic 
forcings).  A  class  of  critical  transitions  in  which  the  state  of  the  system  can  change  suddenly  and 
discontinuously,  known  as  "bifurcations/'  can  occur  in  nonlinear  differential  equations;  for 
example,  old  states  become  unstable  and  disallowed  while  new  stable  states  emerge. 

Unlike  differential  equations,  stochastic  models  allow  variables  to  evolve  in  a  random  rather  than  a 
deterministic  manner.  Also,  these  models  typically  represent  temporal  evolution  discretely  rather 
than  continuously.  Often  the  models  assume  a  Markov  process  in  which  the  probabilities  of  the 
various  future  states  of  the  system  depend  only  on  its  current  state.  In  a  stochastic  simulation  of 
the  epidemic  model  of  ideology  diffusion,  one  could  simulate  all  the  individuals  in  a  population, 
endow  them  with  different  characteristics,  and  allow  infected  individuals  to  meet  and  infect 
susceptible  individuals  with  given  probabilities.  Stochastic  models  are  often  used  in  simulating 
processes  that  occur  over  networks,  such  as  opinion  change  and  cultural  transmission,  and  in 
simulating  network  evolution  itself,  as  in  the  model  of  insurgent  cooperation  presented  below. 
Scale-free  networks  -  characterized  by  power  law  distributions  in  which  a  relatively  few  nodes 
have  a  greatly  disproportionate  degree  (number  of  ties)  -  can  be  generated  by  a  stochastic 
"preferential  attachment"  model  in  which  new  nodes  have  a  higher  probability  of  linking  to  high- 
degree  nodes  than  to  low-degree  ones  (Newman,  2010).  Stochastic  models  can  exhibit  critical 
transitions  resulting  in  abrupt  changes  of  state  in  the  form  of  phase  transitions  at  large  population 
sizes. 

At  its  most  general,  agent-based  modeling  involves  the  specification  of  interaction  rules  for  a 
system  of  agents  in  a  group  or  population.  These  rules  can  be  deterministic  or  probabilistic  and 
expressed  in  terms  of  mathematical  formulas,  similar  to  differential  equations  and  stochastic 
models.  However,  agent-based  modeling  diverges  most  notably  from  these  other  modeling 
approaches  when  the  rules  are  essentially  algorithmic:  specifically,  agent  behavior  is  governed  by  a 
relatively  complex  process  that  accounts  for  strategies,  heuristic  rules,  experience  from  prior 
interactions,  and  learning.  Consequently,  agent-based  models  can  simulate  very  rich  behavior, 
although  the  downside  for  forecasting  purposes  is  that  they  usually  require  more  data  to 
implement  empirically  and  more  computational  resources  to  simulate  the  parameter  space  for 
assessing  potential  outcomes.  Examples  of  applications  of  agent-based  models  to  social  systems  in 
specific  real-world  cases  include  the  modeling  of  social  identity  dynamics  (Alcorn,  Garces,  &  Hicken, 
2012)  and  political  party  competition  (Laver  &  Sergenti,  2012). 


Group  Decision  Making  Model 

To  provide  a  specific  example  of  a  dynamics-based  model  and  a  more  concrete  basis  for  the 
discussion  of  empirical  implementation  issues,  we  now  consider  a  nonlinear  model  of  group 
decision  making  -  one  that  has  been  applied  for  forecasting  purposes  (Gabbay,  2007;  2013).  The 
model  captures  the  evolution  of  group  member  policy  positions  under  the  influence  of  group 
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discussion  and  their  own  ideological  preferences.  Although  the  model  is  formulated  as  a 
differential  equation  system,  it  is  expressed  here  only  in  terms  of  qualitative  rules  to  avoid 
becoming  bogged  down  in  mathematical  detail.  The  rules  are: 

1.  Each  group  member's  current  position  is  affected  by  two  forces:  the  self-bias  force  and  the 
group  influence  force. 

2.  The  self-bias  force  is  the  tension  that  an  individual  feels  when  his  current  position  is 
different  from  his  natural  bias  position,  which  is  determined  by  ideological  and  strategic 
preferences  such  as  the  relative  importance  placed  upon  military,  diplomatic,  economic, 
and  political  factors.  The  self-bias  force: 

a)  Acts  in  the  direction  of  the  natural  bias. 

b)  Has  magnitude  proportional  to  the  difference  between  the  individual's  current 
position  and  natural  bias. 

c)  Increases  in  proportion  to  the  individual's  commitment  to  his  natural  bias. 

3.  The  group  influence  force  is  the  tension  that  an  individual  group  member  feels  when  his 
position  is  different  from  those  of  other  group  members. 

a)  The  total  group  influence  force  on  an  individual  is  the  sum  of  the  coupling  forces 
resulting  from  his  pairings  with  all  the  other  group  members. 

b)  For  a  given  pairing,  the  coupling  force  on  an  individual  acts  in  the  direction  of  the 
position  of  the  other  member  of  the  pair. 

c)  The  magnitude  of  the  coupling  force  increases  approximately  linearly  for  small 
position  differences  but  weakens  for  differences  greater  than  the  individual's  latitude 
of  acceptance. 

d)  The  coupling  force  that  member  j  exerts  on  member  i  is  scaled  by  the  coupling 
strength,  which  characterizes  factors  such  as  how  often  j  communicates  with  /  about 
the  policy  matter  at  hand,  their  relative  status,  /' s  perception  of  j's  credibility  or 
expertise,  and  the  importance  that  /  attaches  to  group  influence. 

At  high  disagreement  levels  (i.e.,  large  differences  in  natural  bias)  the  model  displays  nonlinear 
behaviors,  including  sharp  transitions  between  qualitatively  distinct  outcomes  as  the  coupling 
strength  increases;  asymmetric,  majority  rule  outcomes  resulting  from  symmetric  conditions;  the 
existence  of  multiple  stable  outcomes  for  the  same  parameters;  and  the  greater  facility  of  less 
dense  networks  to  reach  decisions  and  reduce  discord.  These  behaviors  are  not  present  at  low 
disagreement  levels. 


3.  Variables  and  Parameters 

All  dynamics-based  models  are  characterized  by  variables  and  parameters.  Variables  define  the 
state  of  the  system  as  it  evolves  in  the  model  over  time.  In  the  model  above,  the  policy  positions  of 
the  individual  group  members  are  the  state  variables.  Other  examples  of  variables  in  sociocultural 
models  include  cultural  norms,  attitudes,  popular  support  levels  for  government  or  insurgents, 
sizes  of  social  movements,  violence  levels,  and  network  ties  among  individuals  or  organizations.  For 
forecasting  applications,  the  state  variables  should  be  either:  (1)  indirectly  related  to  the  ultimate 
object  of  interest,  in  the  sense  that  the  application  is  intended  to  predict  and  measure  an 
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aggregated  function  of  the  state  variables;  or  (2)  the  objects  of  ultimate  interest  themselves,  in 
that  the  application  is  intended  to  predict  and  measure  the  future  values  of  the  state  variables.  For 
the  group  decision-making  model,  the  first  case  would  apply  if  the  researcher's  ultimate  goal  were 
only  to  forecast  the  final  policy  decision  of  the  group.  This  can  be  done  by  aggregating  the  final 
policy  positions  of  the  individual  group  members  according  to  a  suitable  decision  rule  (e.g.,  leader 
choice,  weighted  averaging,  consensus).  The  second  case  would  apply  if  the  researcher  also  wanted 
to  predict  which  group  members  will  support  the  policy  and  which  ones  will  dissent,  which  can  be 
determined  from  the  distance  between  their  final  position  and  the  final  policy.  Another  example  of 
aggregating  variables,  as  in  (1)  above,  is  a  model  of  insurgency  in  which  the  state  variables  are  the 
attitudes  of  individual  population  members  toward  the  government  and  toward  the  insurgents,  but 
the  application  seeks  to  forecast  only  the  overall  levels  of  support. 

Parameters  are  model  quantities  that  characterize  the  system  exogenously,  i.e.,  they  are  not 
affected  by  the  state  variables.  Parameters  play  a  key  role  in  dynamics-based  models  in 
determining  how  variables  interact  with  each  other  and  the  environment.  In  the  epidemic  model  of 
militant  ideology  spread,  parameters  determine  the  ease  with  which  the  ideology  can  spread 
between  individuals  and  the  rate  at  which  infected  individuals  are  removed  by  government 
repression  or  counterinfluence.  Referring  to  the  group  decision-making  model  above,  the  natural 
bias,  is  used  to  set  the  initial  positions  of  the  members;  the  commitment  scales  the  strength  of  the 
self-bias  force  for  a  given  displacement  of  each  member's  policy  position  from  her  natural  bias;  a 
person's  latitude  of  acceptance  sets  the  range  of  policies  surrounding  her  own  position  that  she 
will  entertain  and  serves  as  the  source  of  the  nonlinear  interactions;  and  the  coupling  strength 
between  individuals  scales  the  impact  of  relational  factors  on  how  effective  a  person  will  be  at 
swaying  another  person's  position.  For  stochastic  models,  the  parameters  determine  how  the 
transition  probabilities  between  states  depend  upon  the  state  variables. 

Modelers  usually  take  parameters  as  constant,  but  can  make  them  time-dependent  as  well.  Given 
the  complexity  of  sociocultural  systems,  variables  often  provide  feedback  to  parameters,  but  as 
long  as  the  time  scale  of  that  feedback  is  slow  relative  to  the  time  scale  at  which  variables  evolve 
one  can  still  consider  the  parameters  as  effectively  exogenous.  If  that  feedback  occurs  at  a  rate 
comparable  to  the  change  in  the  variables,  however,  then  the  modeler  faces  a  coevolution  problem 
and  should  redefine  the  "parameters"  as  state  variables.  The  group  decision-making  model 
example  assumes  that  the  members'  changing  policy  positions  have  no  impact  on  the  coupling 
network  between  group  members.  This  is  reasonable  for  a  relatively  short  decision-making 
episode,  but  if  the  model  were  examining  a  long  time  scale— say  years— then  policy  positions 
would  probably  affect  relationships  under  the  operation  of  the  homophily  principle:  "birds  of  a 
feather  flock  together." 

One  of  the  primary  difficulties  -  perhaps  the  greatest  one  -  in  forecasting  sociocultural  phenomena 
is  that  many  of  the  concepts  used  in  social  science  theory  as  well  as  in  common  discourse  are  not 
readily  quantifiable.  Power,  ideologies,  policies,  attitudes,  beliefs,  grievances,  trust,  relationships, 
disagreements,  utility,  etc.,  do  not  come  in  standard  units  like  watts,  meters,  or  amperes.  For 
instance,  a  social  network  matrix  may  use  a  single  number  to  represent  ties  between  individuals, 
but  that  does  not  capture  the  complexity  of  social  relationships. 
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Forecast,  therefore,  occupies  a  thorny  middle  ground  in  the  abstract -to-concrete  data  spectrum 
between  the  Understand  and  Detect  capabilities.  To  help  understand  general  decision-making 
phenomena,  one  can  model  policy  spaces  in  the  abstract,  but  one  must  assign  actual  policies  to 
policy  space  coordinates  in  order  to  forecast  whether  a  particular  country's  leaders  may  decide 
upon  war  in  a  given  situation.  For  detection,  on  the  other  hand,  very  specific,  concrete,  and  readily 
quantified  data  such  as  the  frequency  of  conversations  may  suffice  to  identify  central  individuals  or 
communities  in,  say,  an  email  network.  However,  that  is  not  equivalent  to  identifying  the  top 
leaders  who  make  strategic  decisions  in  times  of  crisis  and  the  relationships  of  authority  and 
influence  between  them.  Consequently,  a  key  challenge  in  forecasting  is  to  construct  meaningful 
variables  and  parameters  from  relatively  specific,  more  quantifiable,  lower  level  data  that  can  be 
reasonably  mapped  onto  more  abstract,  higher  order  theoretical  concepts. 

This  construction  of  conceptually  broader  variables  and  parameters  is  necessary  not  simply  to 
connect  with  social  science  theory  but  also  to  reduce  the  dimensionality  of  the  space  of  variables 
that  must  be  predicted.  Considering  the  group  decision-making  context,  a  group  member's  natural 
bias  policy  may  be  a  function  of  her  beliefs  about  the  facts,  the  relative  importance  that  she 
accords  to  the  different  components  of  the  problem  -  military,  diplomatic,  economic,  domestic 
political  -  and  how  she  evaluates  the  policy  options  for  each  of  those  components.  Although  an 
analyst  may  be  able  to  assess  those  factors  for  each  group  member,  no  model  can  be  expected  to 
predict  the  changes  in  each  of  those  factors  due  to  group  interactions.  It  is  therefore  necessary  to 
reduce  the  dimensionality  of  the  space  of  the  state  variables  by,  for  instance,  combining  the  above 
factors  to  represent  policy  positions  in  a  one  or  two-dimensional  space.  The  subsections  on  analyst 
input  and  insurgent  rhetoric  present  examples  of  ways  to  construct  ideology  and  policies  from 
lower  level  data. 

Variables  and  some  kinds  of  parameters  can  be  calculated  by  applying  the  defining  formula  or 
algorithm  to  data  representing  a  single  time  instant  or  interval  independently  of  other  times  or 
other  variables.  However,  feedback  parameters  in  particular,  which  necessarily  involve  rates  of 
change  over  time  or  the  strength  of  interaction  among  variables,  cannot  truly  be  calculated  solely 
on  the  basis  of  a  predefined  formula  or  algorithm  that  operates  on  data  representing  only  one 
time.  Ideally,  one  would  measure  the  behavior  of  the  state  variables  over  time  and  then  fit  the 
parameters  to  the  data,  similar  to  the  way  parameters  are  measured  experimentally  in  physical 
systems.  This  is  also  analogous  to  statistically  based  pattern  recognition  and  forecasting  methods  in 
which  an  in-sample  is  used  to  fit  parameters  that  are  then  applied  to  out-sample  data.  Such  a 
fitting  procedure  can  be  used  to  determine  parameters  in  sociocultural  systems,  but  typically  is 
difficult  to  carry  out  to  much  precision  due  to  the  lack  of  sufficient  data  points  over  time  and  the 
relatively  large  amount  of  noise  in  the  data,  among  other  factors.  This  is  particularly  true  of  the 
conflict-ridden  and  chaotic  regions  of  interest  to  the  U.S.  military.  When  fitting  parameters  to 
existing  data  is  not  an  option,  then  modelers  can  choose  parameter  values  in  an  a  priori  manner 
based  on  considerations  of  what  constitutes  reasonable  values  or  ranges.  Simulations  can  then 
sweep  over  a  patch  of  parameter  space  in  order  to  determine  the  sensitivity  of  the  outcomes  to 
changes  in  the  parameters.  Laver  and  Sergenti  (2012)  offer  a  thorough  and  candid  discussion  of  the 
issues  involved  in  parameter  calculation  for  an  agent-based  model  of  political  party  competition. 
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4.  Forecasting  Modes 

Dynamics-based  models  can  be  applied  for  forecasting  purposes  in  a  variety  of  modes.  The  modes 
available  for  a  given  application  depend  on  the  nature  of  the  data.  One  mode  involves  predicting, 
in  a  probabilistic  way,  the  evolution  of  variables  that  can  change  in  a  more-or-less  continuous 
fashion  over  time:  for  instance,  the  levels  of  popular  support  for  an  opposition  movement  among 
different  segments  of  the  population.  This  would  correspond  to  true  forecasting  in  that  one  assigns 
time-dependent  probabilities  of  the  various  possible  states  of  the  system,  as  is  done  in  weather 
forecasting.  However,  as  discussed  above,  this  remains  an  ambitious  goal  for  most  models  of 
sociocultural  dynamics  due  to  the  limitations  in  measuring  variables  and  parameters  (as  well  as 
model  uncertainty  itself,  a  topic  not  considered  in  this  chapter).  Although  it  would  be  technically 
possible  to  develop  simulation  software  that  makes  explicitly  probabilistic  forecasts,  doing  so  could 
well  be  counterproductive  in  the  absence  of  accurate  and  precise  estimates  of  model  variables  and 
parameters.  Otherwise,  end  users,  accustomed  to  reliable  weather  forecasts,  could  be  lulled  into  a 
false  sense  of  confidence  by  the  simulation's  provision  of  numerical  probabilities  and, 
consequently,  might  be  greatly  disappointed  by  the  inevitable  large  mismatch  between  the 
forecasts  and  reality. 

A  looser  mode  of  forecasting  can  be  applied  to  key  events,  such  as  conflicts,  decisions,  and  alliance 
composition,  which  can  be  treated  as  one-off  occurrences.  In  these  instances,  one  relaxes  the 
requirement  that  the  model  make  forecasts  over  time  and  simply  assumes  that  the  time  window  is 
sufficiently  long  to  allow  the  system  to  reach  equilibrium.  In  the  group  decision-making  model,  the 
time  scales  that  enter  into  the  commitment  and  coupling  scale  parameters  are  not  usually  known, 
which  precludes  such  a  model  from  being  used  to  forecast  the  evolution  of  the  group  members' 
policy  positions  in  real  units  of  time.  However,  the  equilibrium  positions  themselves  do  not  depend 
on  the  time  scales,  and  so  the  model  can  still  be  used  to  predict  the  final  decision  and  its 
supporters  and  dissenters,  on  the  assumption  that  the  debate  persists  until  equilibrium.  This 
equilibrium  approach  significantly  eases  data  demands;  many  dynamics-based  models  use  this 
mode  in  practice. 

Sensitivity  analyses  in  which  simulations  cover  a  region  in  the  parameter  space  can  also  be 
employed  for  forecasting  purposes.  If  the  outcome  does  not  vary  significantly  as  parameters  sweep 
over  a  range  of  reasonable  or  likely  values,  then  the  model  user  can  conclude  that  the  outcome  is 
highly  probable.  Alternatively,  the  simulations  may  reveal  a  range  of  possible  outcomes.  These  may 
include  potential  outcomes  (e.g.,  alliance  configurations)  not  anticipated  prior  to  the  simulation. 
Sensitivity  analysis  may  also  reveal  that  the  outcome  can  change  dramatically  with  only  a  small 
shift  in  parameter  values,  as  would  be  consistent  with  a  bifurcation  or  phase  transition. 

As  an  illustration,  consider  a  group  decision-making  model  in  which  all  group  members  are 
connected  with  the  same  coupling  strengths  and  have  initial  opinions  (natural  biases) 
symmetrically  distributed  around  zero.  More  specifically,  consider  a  three-person  group  in  which  a 
centrist  is  bracketed  by  two  equidistant  extremists.  Standard  intuition  would  anticipate  either  a 
deadlock  or  various  shades  of  compromise  around  the  centrist  position,  consistent  with  final  states 
that  are  symmetric  about  the  middle,  as  shown  in  Figure  1  (a)  and  (b)  for  deadlock  and  compromise 
respectively.  However,  at  sufficiently  high  levels  of  initial  disagreement,  another  outcome  can 
result  in  which  the  centrist  swings  toward  one  of  the  extremes  (depending  on  random 
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perturbations),  corresponding  to  a  majority  rule  situation  favoring  one  side  of  the  policy  axis.  In 
this  case  the  system  reaches  an  asymmetric  final  state  as  observed  in  Figure  1(c)  (Gabbay  &  Das, 
2014).  Even  a  small  change  in  coupling  strength  can  produce  a  transition  to  the  majority  rule 
outcome  zone  from  either  the  deadlock  or  compromise  zones.  If  the  simulation  of  a  leadership 
group  showed  such  behavior,  analysts  could  be  prepared  for  an  extreme  decision  rather  than 
deadlock  or  compromise. 


(a)  Deadlock  (b)  Compromise 


(c)  Majority  Rule 


Figure  1.  Group  decision-making  simulation  of  member  positions  over  time  in  triad  network  (chain 
topology)  with  symmetric  coupling  strengths  and  natural  bias  distributions:  (a)  symmetric  deadlock 
outcome  at  low  coupling  strengths;  (b)  symmetric  compromise  outcome  at  high  coupling  strengths; 
and  (c)  asymmetric  majority  rule  outcome  at  intermediate  coupling  strengths.  Adapted  with 
permission  from  Gabbay  &  Das  (2014). 


Finally,  dynamics-based  models  can  be  used  to  conduct  scenario  analyses  to  forecast  the  effects  of 
specific  hypothesized  contingencies  or  conditions.  These  analyses  can  be  implemented  by 
particular  variable  or  parameter  settings  that  represent  exogenous  impacts  on  the  system  due  to 
events  or  actions  outside  the  dynamics  of  the  model.  For  example,  in  a  group  decision-making 
situation,  one  can  simulate  the  effect  upon  the  policy  outcome  of  a  rupture  in  the  relationship 
between  two  group  members  due  to  a  personal  dispute  unrelated  to  the  policy  context.  This  could 
be  done  by  nulling  or  severely  attenuating  their  mutual  coupling  strengths,  possibly  in  combination 
with  other  parameter  changes. 


5.  Analyst  Input  for  Group  Decision  Making 

The  two  principal  sources  of  data  for  calculating  the  "soft"  variables  and  parameters  (opinions, 
beliefs,  ideologies,  norms,  social  identities,  etc.)  needed  as  model  input  are  analyst  (subject  matter 
expert)  judgments  and  content  analysis.  Currently,  analyst-based  input  represents  the  prevalent 


Data  processing  for  applications  of  dynamics-based  models  to  forecasting  |  253 


form  of  input  (e.g.,  Alcorn  et  al.,  2012;  Feder,  2002;  Gabbay,  2013),  but  the  use  of  content  analysis- 
based  input  is  growing,  especially  given  recent  advances  in  automated  text  analysis.  This  section 
briefly  discusses  the  use  of  analyst  input  in  conjunction  with  the  group  decision-making  model 
described  above.  The  next  section  describes  an  application  of  content  analysis  using  rhetoric  as 
input  data  for  the  context  of  insurgent  network  dynamics. 

Figure  2  shows  the  processing  chain  to  implement  the  model  for  specific  leadership  groups  using 
analyst  judgment  for  input.  The  modeler  obtains  this  input  from  a  survey  given  to  one  or, 
preferably,  many  analysts.  A  composite  analyst  can  be  formed  by  averaging  the  survey  responses 
of  the  individual  analysts.  The  aggregation  of  individual  surveys  allows  analyst  judgments  to  be 
synthesized  independently  of  each  other.  This  minimizes  the  chance  that  social  pressure  will  alter 
individual  judgment,  as  can  happen  if  the  modeler  elicits  inputs  in  an  oral  discussion  with  a  group 
of  analysts  -  a  common  practice  in  other  models  of  group  decision-making  used  within  the  national 
security  community  (Bueno  de  Mesquita,  2009).  Note  that  results  can  be  generated  on  the  basis  of 
individual  surveys  as  well.  This  permits  comparison  of  the  results  obtained  from  individual  analysts 
with  the  composite  analyst  and  with  each  other,  thereby  providing  a  way  of  stimulating  debate 
about  differences  between  analyst  viewpoints. 


Structural  Analysis 


Survey  Responses 


Analyst  #1 

'.v  ‘  "  | 

Survey 

Aggregation 


Problem 

Definition/ 
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Generation 
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<=> 
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Parameter 
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Analyst  #N 

Group  Decision 
Making  Simulation 


Figure  2.  Overview  of  analyst  input-based  implementation  of  group  decision-making  model. 
Adapted  with  permission  from  Gabbay  (2013).  Copyright  held  by  the  Institute  for  Computer 
Sciences,  Social  Informatics  and  Telecommunications  Engineering. 


254  |  Data  processing  for  applications  of  dynamics-based  models  to  forecasting 


Another  innovation  in  this  implementation  methodology  is  the  use  of  an  attitude  scaling  technique 
to  assess  member  ideological  and  policy  positions.  The  natural  bias,  which  is  typically  taken  as  the 
initial  position  of  a  group  member,  is  constructed  on  the  basis  of  a  series  of  attitude  statements  for 
which  the  analyst  rates  the  level  of  agreement  or  disagreement  as  if  she  were  the  group  member  in 
question,  as  shown  in  Table  1.  This  enables  assessment  of  several  different  facets  of  the  policy 
problem  that  can  then  be  aggregated  to  calculate  the  natural  bias.  Also,  modelers  can  probe  the 
deeper  structure  of  the  policy  space  by  using  matrix  decomposition  methods  such  as  principal 
component  analysis.  This  approach  elicits  analysts'  expertise  on  group  member  policy  preferences 
without  demanding  that  they  directly  perform  the  abstraction  needed  to  create  a  policy  axis  or 
space  itself  —  a  task  that  the  modeler  accomplishes  instead. 


Table  1.  Sample  attitude  statements  assessing  ideological  and  policy  preferences  from  an  analyst 
survey  on  Afghan  government  elites.  Analysts  are  asked  to  rate  member  attitudes  on  a  five-point 
scale  ranging  from  "strongly  disagree"  to  "strongly  agree" 


# 

Statement 

Karzai 

Fahim 

Khalili 

Rabbani 

Dostum 

z 

c 

1 

Partition  of  Afghanistan  should  be  considered  to  end  the  conflict, 
if  necessary. 

2 

Afghanistan  should  have  a  federal  system  of  government 
where  regions  have  effective  autonomy  to  govern  themselves. 

3 

Karzai's  efforts  to  concentrate  power  in  the  presidency  show  that 
the  Afghan  Constitution  should  be  changed  to  institute  a 
parliamentary  -centered  system  of  government. 

4 

A  strong  central  government  is  needed  in  order  to  hold 

Afghanistan  together. 

5 

A  strong  leader  with  power  concentrated  in  his  hands  is  needed 
to  rule  Afghanistan. 

6 

The  regions  in  a  divided  Afghanistan  would  become  weak 
vassals  of  neighboring  countries  like  Iran  and  Pakistan. 

7 

It  is  important  to  preserve  the  integrity  of  Afghanistan  as  a 

multi-ethnic  state. 

8 

It  is  the  historic  right  of  Pashtuns  to  lead  the  Afghan  state. 
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6.  Rhetoric  Input  for  Insurgent  Network  Dynamics 

This  section  presents  an  application  using  input  derived  from  content  analysis  -  both  automated 
and  manual  -  in  which  the  data  source  is  Iraqi  insurgent  rhetoric.  It  describes  methods  for 
quantifying  insurgent  ideology,  targeting  policy,  and  cooperative  relationships.  Those  three 
elements  roughly  correspond  to  insurgent  ends,  means,  and  allies  respectively:  critical  components 
of  their  strategic  behavior  and  decision  making.  These  elements  can  be  used  to  track  insurgent 
behavior  over  time  and  as  inputs  in  models  for  forecasting  insurgent  dynamics.  A  model  of 
insurgent  tactical  cooperation  is  presented  at  the  end  of  this  section  as  an  example  of  how  such 
data  can  be  used  for  simulating  insurgent  network  evolution. 

The  modelers  collected  data  for  the  18  Iraqi  Sunni  insurgent  groups  listed  in  Table  2,  spanning  the 
time  from  mid-2003  through  mid-2009.  This  time  span  is  divided  into  three  periods:  (1)  Period  1, 
August  2003  -  July  2005;  (2)  Period  2,  August  2005  -  July  2007;  and  (3)  Period  3,  August  2007  -  July 
2009.  The  data  set  consists  of  roughly  2,000  translated  insurgent  statements  from  jihadist  websites 
and  interviews  of  insurgent  group  officials  in  print  and  broadcast  media  as  provided  by  the  U.S. 
government's  Open  Source  Center. 
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Table  2.  Sunni  insurgent  groups  in  Iraq  used  in  content  analysis  of  insurgent  rhetoric 


Group 

Symbol 

Overall 

Classification 

Islamist 

Ideology 

Time 

Periods 

Al-Qaida  in  Iraq 

AQI 

Jihadist  Salafist 

Salafist 

1,2,3 

Ansar  al-Sunnah  Army 

(Ansar  al-lslam:  post-Dec.  2007) 

ASA 

Jihadist  Salafist 

Salafist 

1,2,3 

Islamic  Army  in  Iraq 

IAI 

Nationalist 

Salafist 

1,2,3 

Mujahidin  Army 

MA 

Nationalist 

Salafist 

1,2,3 

1920  Revolution  Brigades 

1920RB 

Nationalist 

Unspecified 

1,2,3 

Islamic  Front  for  Iraqi  Resistance 

JAMI 

Nationalist 

Muslim 

Brotherhood 

1,2,3 

Rashidin  Army 

RA 

Nationalist 

Unspecified 

1,2,3 

HAMAS-lraq 

HAMI 

Nationalist 

Muslim 

Brotherhood 

3 

Fatihin  Army 

FA 

Nationalist 

Salafist 

2,3 

Iraqi  Jihadist  Leagues 

IJL 

Jihadist  Salafist 

Salafist 

2,3 

Shield  of  Islam  Brigade 

SIB 

Jihadist  Salafist 

Salafist 

2,3 

Ansar  al-Sunnah  Shariah 

Commission 

ASA-SH 

Nationalist 

Salafist 

3 

Just  Punishment  Brigades 

JPB 

Jihadist  Salafist 

Salafist 

2,3  (part) 

Abu  Bakr  al-Siddiq  Salafi  Army 

ABSSA 

Jihadist  Salafist 

Salafist 

2,3 

Islamic  Jihad  Brigades 

ISJIBR 

Jihadist  Salafist 

Salafist 

1,2 

Victorious  Sect  Army 

VSA 

Jihadist  Salafist 

Salafist 

1,2 

Saad  Bin  Abi  Waqqas  Brigades 

SBAW 

Jihadist  Salafist 

Salafist 

2,3 

Army  of  Naqshabandi  Order 

NAQSH 

Nationalist 

(Baathist) 

Sufi 

2,3 

Ideology 

Ideology  is  quantified  by  the  concept  of  a  conflict  frame.  The  conflict  frame  of  a  political  actor,  such 
as  an  insurgent  group  or  government  leaders,  is  defined  on  the  basis  of  the  set  of  out-group  parties 
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perceived  as  current  or  potential  enemies  and  in-group  parties  perceived  as  allies  or  as  a  base  of 
support  (Gabbay  &  Thirkill-Mackelprang,  2011).  This  definition  stems  from  social  identity  theory 
and  its  implications  for  political  rhetoric.  Conflict  parties  consist  of  broad  groups  such  as  the 
incumbent  government,  ethnic  or  religious  groups  and  their  leadership  classes,  and  foreign  states. 
The  mathematical  formalism  for  conflict  frames  relies  on  the  frequency  with  which  specified 
marker  terms  appear  in  actor  rhetoric  and  their  in/out-group  valences.  Figure  3  is  a  conceptual 
diagram  of  the  elements  involved  in  the  conflict  frame  calculation  procedure.  Sample  marker  terms 
and  associated  in-  and  out-groups  are  given  in  Table  3. 


GROUP 

RHETORIC 


Figure  3.  Conceptual  diagram  of  conflict  frame  method  of  quantifying  ideology. 
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Table  3.  Examples  of  marker  terms  and  associated  in-groups  and  out-groups  for  Iraqi  insurgents 


Marker  Term 

In-Groups 

Out-Groups 

agent  government 

Iraqi  Government 

apostate  government 

Sunni  Civil  Society 

Iraqi  Government 

companions  of  the  prophet 

Sunni  Civil  Society 

cross  worshippers 

Sunni  Civil  Society 

United  States 

Iraqis 

Iraqi  Civil  Society 

Iranian  occupation 

Sunni  Civil  Society 

Iraqi  Government,  Shiite  Political 

Parties 

occupier 

United  States 

rejectionist 

Sunni  Civil  Society 

Shiite  Civil  Society 

rescue  council 

Sunni  militias  (Awakening  Councils) 

Each  marker  term  is  assigned  a  valence  for  all  the  conflict  parties  to  which  it  refers:  positive  (+1)  for 
in-groups,  negative  (-1)  for  out-groups,  and  zero  for  neutral  references.  A  conflict  party's  salience 
to  the  actor  is  essentially  the  frequency  with  which  marker  terms  referring  to  that  party  appear  in 
the  actor's  rhetoric,  regardless  of  valence.  The  actor's  attitude  toward  the  party  is  related  to  the 
relative  frequency  of  positive  and  negative  references  to  it.  The  attitude  weighted  by  the  salience 
represents  the  actor's  orientation  toward  the  party.  An  actor  can  have  a  highly  negative  attitude 
toward  a  conflict  party  but  a  low  orientation  value  if  the  actor  seldom  refers  to  the  party,  which 
hence  has  low  salience.  The  collection  of  orientation  values  for  all  the  conflict  parties  forms  the 
conflict  frame  of  the  insurgent  group. 

This  procedure  is  automated  following  the  construction  of  the  initial  dictionary  and  valence  matrix 
and  therefore  can  be  updated  in  near-real  time  to  track  ideological  shifts  and  divisions  within  the 
actor  ensemble.  An  actor's  conflict  frame  vector,  composed  of  the  actor's  orientations  toward  the 
conflict  parties,  is  the  primary  output  of  the  algorithm.  Another  way  of  analyzing  the  data, 
however,  is  by  considering  certain  master  frames,  which  can  be  given  intuitive  interpretations.  A 
master  frame  consists  of  a  specified  subset  of  out-groups  and  in-groups.  The  extent  to  which  an 
actor  espouses  a  given  master  frame  can  be  quantitatively  gauged  by  how  closely  its  conflict  frame 
aligns  with  the  master  frame  -  mathematically  via  the  inner  product  of  the  corresponding  vectors. 
Two  master  frames  dominate  for  the  Iraqi  insurgents:  (a)  the  resistance  frame  that  pits  Iraqis  as  in¬ 
group  against  the  United  States  as  out-group  and  (b)  the  sectarian  frame  that  pits  Sunnis  as  in¬ 
group  against  Shiites  as  out-group. 
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Figure  4  shows  the  conflict  frames  for  the  Iraqi  insurgent  groups  over  the  three  time  periods.2  The 
Jihadist  groups— particularly  the  two  major  ones,  Al  Qaida  in  Iraq  (AQI)  and  Ansar  al-Sunnah— tend 
to  fall  on  the  high  end  of  the  sectarian  frame  and  the  low  end  of  the  resistance  frame.  The 
Nationalists,  especially  "pure"  Nationalist  groups  such  as  the  1920  Revolution  Brigades,  JAMI,  and 
the  Rashidin  Army,  have  lower  sectarian  frame  values  and  high  resistance  values.  However,  it  is 
important  to  note  that  the  figures  show  a  spectrum  rather  than  two  widely  disjoint  clusters  of 
Jihadists  and  Nationalists,  as  would  be  implied  by  such  a  binary  analytical  classification.  This 
spectrum  quality  and  the  locations  of  individual  groups  in  the  master  frame  coordinate  space  have 
meaningful  implications  for  cooperative  behavior  among  insurgent  groups.  The  first  two  Nationalist 
alliances  to  nucleate  comprised  neighbors  in  the  Period  2  master  frame  space:  (1)  the  Jihad  and 
Reform  Front  consisting  of  I  Al,  MA,  FA,  and  ASA-SH;  and  (2)  the  Jihad  and  Change  Front  consisting 
of  1920RB  and  RA.  This  suggests  that  the  conflict  frame  construct  can  provide  meaningful  and 
predictive  input  to  models  of  insurgent  alliance  formation. 


2The  groups  ABSSA  and  ISJIBR  are  not  included  (documents  were  not  preprocessed  to  remove  extraneous 
text). 
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(a)  Aug03-Jul05 


(b)  Aug05-Jul07 


(c)  Aug07-Jul09 


Figure  4.  Master  frames  for  Iraqi  insurgent  groups:  (a)  Aug03-Jul05;  (b)  Aug05-Jul07;  (c)  Aug07- 
Jul09.  Jihadists  in  black  squares,  Nationalists  in  white  circles. 
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Tracking  changes  is  one  of  the  objectives  of  the  Forecast  area  and  the  time-dependent  behavior  of 
the  conflict  frames  can  be  tracked  for  either  individual  groups  (as  in  Figure  4)  or  the  whole 
ensemble.  For  example,  in  tracking  changes  in  the  balance  between  the  two  master  frames  for 
whole  ensemble  of  groups,  Figure  5(a)  shows  that  the  sectarian  frame  component  increases 
relative  to  the  resistance  frame  from  the  first  to  the  second  period,  even  among  the  Nationalists, 
and  then  wanes  again  in  the  third.  This  tracks  the  transition  into  a  Sunni-Shiite  civil  war  during  2006 
and  2007  and  the  subsequent  decline  in  sectarian  violence.  Ideological  dissension  within  the 
insurgency  can  be  tracked  by  looking  at  the  frame  deviation:  i.e.,  how  spread  out  the  groups  are  in 
the  space  of  master  frames.  Figure  5(b)  shows  the  frame  deviation  in  the  resistance-sectarian 
master  frame  space.  The  deviation  starts  out  low  during  the  first  period  but  increases  and  plateaus 
for  the  second  and  third  periods.  This  corresponds  to  the  character  of  factional  relations  within  the 
insurgency.  Early  on,  the  groups  were  relatively  united,  but  the  transition  to  a  sectarian  civil  war 
was  a  key  factor  in  producing  a  rift  between  the  Jihadist  and  Nationalist  wings  of  the  insurgency, 
eventually  leading  to  the  formation  of  the  anti-AQI  Awakening  Councils  and  open  fighting  between 
Nationalist  groups  and  AQI. 


(a) 


Aug03-Jul05  Aug05-Jul07  Aug07-Jul09 

Time  Period 


(b) 


Time  Period 

Figure  5.  Temporal  tracking  measures  for  Iraqi  insurgent  ideology:  (a)  Ratio  of  mean  sectarian 
frame  value  to  mean  resistance  frame  value;  (b)  Standard  deviation  in  group  two-dimensional 
frame  positions  from  mean  position  of  sectarian  and  resistance  frames. 
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Targeting  Policy 

To  construct  the  targeting  policy  variable,  the  portfolio  of  target  classes— U.S.  troops,  Iraqi  security 
forces,  Shiite  militias,  government  officials,  civilians,  etc.— claimed  by  different  insurgent  groups  is 
considered  (Gabbay  &  Thirkill-Mackelprang,  2011).  The  targeting  policy  scores  each  insurgent 
group  by  the  average  legitimacy  of  its  target  class  portfolio,  where  the  "legitimacy"  of  each  target 
class  is  the  acceptability  of  attacking  it  as  determined  by  the  prevalence,  within  the  set  of  insurgent 
groups,  of  claims  and  statements  supporting  targeting  the  class  vs.  condemnations  of  doing  so.  A 
high  targeting  policy  corresponds  to  a  more  discriminate  or  selective  use  of  violence,  while  a  low 
one  indicates  more  expansive  and  controversial  targeting  practices.  Implicit  in  this  construction  is 
that  distance  along  the  targeting  policy  axis  is  related  to  the  extent  of  disagreement.  This  reflects 
that  disagreement  over  the  legitimacy  of  different  types  of  targets  has  often  been  the  primary 
source  of  dissension  within  Islamist  insurgencies  (Hafez,  2003). 


Figure  6  shows  insurgent  group  targeting  policies  over  the  second  and  third  time  periods.  The 
groups  are  arranged  along  the  vertical  axis  in  ascending  targeting  policy  order  so  that  groups  at  the 
bottom  have  lower  targeting  policies.  The  figure  shows  that  the  Jihadists  tend  to  have  lower 
targeting  policies  and  Nationalists  higher  ones.  However,  similar  to  conflict  frames,  this  reflects  a 
spectrum  rather  than  a  binary  distribution.  Like  conflict  frames,  targeting  policy  can  also  be  used  to 
track  dissension  among  insurgent  factions;  the  standard  deviation  of  the  targeting  policy  increases 
over  time  in  tandem  with  the  actual  rise  in  dissension  within  the  insurgency  (Gabbay  &  Thirkill- 
Mackelprang,  2011). 
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Figure  6.  Targeting  policies  for  Iraqi  insurgent  groups:  (a)  Aug05  -  Jul07;  (b)  Aug07  -  Jul09.  Jihadists 
in  gray,  Nationalists  in  white. 


Cooperative  Networks 

The  rhetoric-based  methodology  constructs  networks  representing  cooperative  relationships 
among  insurgents  at  both  the  leadership  and  rank-and-file  levels.  Leadership  relationships  among 
groups  are  gauged  by  the  number  of  joint  communiques  they  issue.  A  joint  communique  is  a 
statement  signed  by  two  or  more  groups,  indicating  the  presence  of  communication  and  some 
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level  of  agreement  among  the  leadership  of  the  groups  issuing  it.  Furthermore,  it  demonstrates  a 
willingness  of  the  groups  involved  to  be  publicly  associated  with  each  other.  At  the  rank-and-file 
level,  the  method  uses  the  number  of  joint  operations  between  groups.  Typically,  only  one  of  the 
participants  makes  a  claim  of  joint  operations.  Such  operations  indicate  tactical  coordination 
among  insurgent  groups,  although  presumably  a  group's  leadership  would  have  to  approve  the 
public  disclosure  of  such  cooperation.  Figure  77  (a)  shows  the  joint  operations  network  for  the 
third  time  period;  Figure  77  (b)  shows  the  simulated  version,  which  is  discussed  presently. 


(a)  Observed  Network 


(b)  Simulated  Network 


max  correlation  axis 


Figure  7.  Joint  operations  networks  for  Aug07  -  Jul09:  (a)  observed;  (b)  simulated.  Jihadists  in  black 
squares,  Nationalists  in  white  circles. 


Application  to  Modeling  Insurgent  Tactical  Cooperation 

The  elements  of  the  above  representation  can  serve  as  input  data  for  a  stochastic  model  of 
insurgent  tactical  cooperation  (Gabbay  &  Thirkill-Mackelprang,  2010).  The  model  variables  are  the 
numbers  of  joint  operations  between  insurgent  group  pairs,  whereas  targeting  policy  and  the 
leadership  network  ties  are  taken  as  fixed  parameters.  The  model  assumes  that  each  insurgent 
group  has  a  number  of  foot  soldier  field  units  that  may  cooperate  with  the  units  of  other  groups. 
The  model  describes  the  probability  that  some  unit  from  group  /  will  conduct  a  joint  operation  with 
some  unit  from  group  j.  It  assumes  that  the  joint  operation  process  depends  on  the  group  sizes,  the 
number  of  prior  joint  operations  between  units  of  /  and  j,  their  tendency  to  interact  with  similar 
other  groups,  their  targeting  policies,  and  the  presence  or  absence  of  leadership  relationships. 
Figure  8  shows  a  diagram  of  the  process. 
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Figure  8.  Simulated  process  for  evolution  of  a  joint  operations  network. 


An  important  feature  of  this  model  is  its  positing  of  roles  for  both  horizontal  interaction  in  which 
foot  soldier  units  can,  on  their  own  initiative,  cooperate  with  units  from  other  groups  and  a 
hierarchical  process  through  which  group  leaderships  can  decide  whether  or  not  to  let  such 
operations  go  forward  and/or  claim  responsibility  for  them  if  they  occurred.  This  horizontal  and 
hierarchical  blending  manifests  the  hybrid  nature  of  insurgent  groups  that  mix  formal  and  informal 
organizational  structures. 

Figure  7  displays  a  comparison  of  the  observed  and  simulated  networks  for  the  third  time  period. 
The  simulated  network  represents  the  average  over  500  simulation  runs.  The  simulation  used  the 
state  of  the  joint  operations  network  in  the  previous  period  to  represent  the  initial  conditions,  but 
used  the  targeting  policies  and  leadership  network  of  the  third  period  to  better  assess  the  tactical 
cooperation  model  itself.  Model  parameter  values  (other  than  the  fixed  targeting  policies  and 
leadership  networks)  were  chosen  based  on  intuitively  reasonable  estimates  and  set  to  be  the 
same  for  the  simulations  of  the  second  and  third  period  networks;  some  variation  of  parameters 
was  conducted  but  an  extensive  search  of  the  parameter  space  to  optimize  results  was  not 
performed.  The  visualizations  show  good  agreement  in  the  placement  of  the  insurgent  groups;  only 
MA  and  RA  are  significantly  out  of  place  in  the  simulation  (readers  should  ignore  the  links  in  the 
simulation  plot  as  they  have  been  thresholded  to  unclutter  the  graphic).  For  forecasting  purposes, 
analysts  could  employ  such  a  model  to  predict  changes  in  patterns  of  insurgent  cooperation  under 
different  scenarios,  such  as  increased  dissension  over  targeting  practices. 


7.  Future  Research  Directions 

This  section  identifies  some  current  shortcomings  in  existing  capabilities  for  data  processing  in  the 
Forecast  capability  area  and  suggests  future  research  directions.  By  necessity,  the  discussion  is 
selective  and  emphasizes  areas  related  to  the  types  of  application  domains  presented  in  this 
chapter.  It  also  stresses  the  need  to  develop  data  appropriate  for  dynamics-based  models, 
advocates  more  dedicated  efforts  to  collect  and  process  data  on  elites  and  their  organizations,  and 
suggests  the  development  of  multi-level  network  representations  as  one  avenue  for  structuring 
this  data. 

Data  processing  is  a  critical  enabling  capability  in  the  application  of  dynamics-based  models  for 
forecasting  purposes.  Better  data  will  naturally  lead  to  better  models.  While  this  is  true  of  both 
statistical  and  dynamics-based  models,  the  latter  ultimately  hold  more  promise.  Statistical  methods 
are  powerful  because  they  often  can  be  applied  to  data  in  a  context-independent  way,  but  they 
have  serious  limitations  when  applied  to  the  contexts  of  particular  concern  to  national  security 
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analysts,  planners,  and  policy  makers— situations  often  marked  by  crisis,  conflict,  and  chaos.  For 
example,  regression  models  can  endeavor  to  forecast  future  violence  levels  in  a  conflict  using  only 
a  time  series  of  past  violence  levels  without  any  need  to  know  about  the  power  balance, 
constraints,  and  internal  dynamics  of  the  warring  parties.  However,  this  results  in  futures  that 
hardly  differ  from  the  recent  past.  Regression  models,  for  instance,  would  never  have  anticipated 
the  precipitous  drop  in  violence  in  Iraq  due  to  the  synergistic  effects  of  the  Sunni  Awakening  and 
the  U.S.  troop  "surge"  in  2007,  when  the  Sunnis,  squeezed  on  both  sides  by  the  extremist  AQI  and 
the  newly  empowered  Shiite  majority,  decided  to  side  with  the  United  States  against  AQI  (Biddle, 
Friedman,  &  Shapiro,  2012).  Given  that  most  forecasting  of  social  and  political  systems  occurs 
within  a  statistical  modeling  framework,  more  focused  research  is  needed  on  aligning  data 
processing  with  dynamics-based  models. 

It  is  not  only  the  availability  of  more  data  that  is  important  but  also  the  intelligent  processing  of 
that  data  into  quantities  that  are  (a)  theory-driven  and  (b)  operationally  relevant.  The  first  criterion 
refers  to  the  need  for  compact,  low-dimensional  quantitative  representations  that  correspond  to 
theoretical  constructs  from  social  science.  The  second  refers  to  the  need  to  include  variables  and 
parameters  that  can  change  on  time  scales  of  operational  concern  or  can  be  influenced  on  those 
time  scales  using  operationally  available  means.  Imposing  these  criteria  will  facilitate  model 
development  and  the  practical  application  of  computational  models  to  forecasting.  In  such 
applications  the  models  will  address  the  issues  most  relevant  to  intelligence  analysts  and 
operational  planners  (thus  tying  into  the  Mitigate  capability  area)  and  can  be  used  to  forecast 
changes  over  a  relatively  small  number  of  dimensions.  Operational  relevance  has  not  been  a 
strength  of  most  extant  instability  forecasting  work  that  relies  on  structural  factors  such  as  low 
economic  development,  regime  type,  ethnic  fragmentation,  mountainous  terrain,  existence  of  a 
territorial  dispute,  etc.  These  factors  may  predispose  a  country  to  conflict  but  indicate  little  about 
the  situation-specific  triggers,  timing,  and  levers  of  influence  that  may  exacerbate  or  ameliorate  a 
conflict  situation  (O'Brien,  2010). 

The  insurgent  network  application  presented  above  illustrates  a  theory-driven  quantitative 
representation  of  soft,  intangible  variables  for  organizations.  The  definition  of  insurgent  targeting 
policy  -  a  one-dimensional  measure  -  is  motivated  by  social  science  research  showing  the 
importance  of  disagreement  over  targeting  practices,  such  as  indiscriminate  attacks  against 
civilians,  in  the  relations  between  insurgent  groups.  The  application  of  social  identity  theory  to 
political  rhetoric  prompted  the  use  of  the  conflict  frame  as  an  ideology  variable.  The  theory's 
definition  in  terms  of  in-groups  and  out-groups  makes  the  data  well  suited  to  serve  as  input  to 
models  of  alliance  dynamics,  as  its  correspondence  to  the  formation  of  alliances  among  Iraqi 
insurgent  groups  makes  clear.  It  can  also  be  an  input  to  models  of  insurgent  strategic  violence  and 
is  in  fact  correlated  with  targeting  policy  (Gabbay  &  Thirkill-Mackelprang,  2011).  The  number  of 
parties  to  the  conflict  gives  the  dimension  of  the  conflict  frame  vector,  which  can  be  further 
reduced  by  projection  onto  a  master  frame  space,  as  described  earlier  in  this  chapter.  Both 
targeting  policy  and  conflict  frames  have  operational  relevance  because  they  influence  and  reflect 
significant  changes  in  the  nature  of  the  conflict  itself  and,  hence,  evolve  on  operational  time  scales. 
In  addition,  they  can  inform  analysis  and  planning  of  operational  means  such  as  information 
operations  or  selection  of  particular  groups  for  targeting  or  negotiations. 
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Additional  types  of  data  and  improved  processing  techniques  focused  on  elites  and  their 
organizations  -  whether  in  the  government  or  its  opposition,  at  a  national  or  a  local  level  -  would 
be  particularly  helpful  in  developing  theory-driven  and  operationally  relevant  forecasting  methods. 
The  decisions  and  actions  of  these  individuals  and  groups  directly  and  immediately  shape  the 
trajectories  of  crises  and  conflicts.  The  recent  surge  of  research  on  popular  opinion,  driven  by  the 
availability  of  social  media  and  automated  tools  for  sentiment  analysis,  has  overshadowed  the 
significance  of  elites  and  their  organizations.  Although  generic  sentiment  analysis  software  could 
be  applied  to  the  rhetoric  of  political  leaders  and  organizations,  it  would  likely  not  provide  a  useful 
data  processing  tool  for  forecasting  their  behavior.  Standard  sentiment  analysis  methods,  originally 
developed  for  contexts  such  as  movie  and  consumer  product  reviews,  have  not  proven  to  work 
well  on  the  discourse  of  political  elites,  whose  rhetoric  must  often  be  circumspect  and  cast  in 
neutral  language,  relying  more  on  nouns  and  topic  emphasis  than  on  adjectives,  which  carry 
positive  or  negative  sentiment  in  ordinary  speech  (Yu,  Kaufmann,  &  Diermeier,  2008).  However, 
researchers  have  used  other  types  of  text  mining  algorithms,  such  as  support  vector  machines,  to 
successfully  classify  elite  political  rhetoric  (Diermeier,  Godbout,  Yu,  &  Kaufmann,  2012). 

In  support  of  this  call  for  more  research  on  elite  and  organizational  data,  it  has  been  found  that  the 
presence  of  factionalism  within  a  state  plays  a  very  significant  role  in  forecasting  political  instability 
via  statistical  methods  (Goldstone  et  al.,  2010).  This  implies  that  accurate  and  timely  metrics  of 
elite  or  organizational  factional  dissension,  like  the  conflict  frame-based  measure  of  ideological 
dissension  in  Figure  5(b),  should  improve  forecasting  of  crises  such  as  coups,  rebellions,  and  civil 
wars.  Researchers  may  find  it  particularly  worthwhile  to  integrate  data  on  elites  with  political  event 
data  that  encode  interactions  such  as  violence  between  political  actors.  Automated  methods  for 
extracting  event  data  from  news  feeds  have  been  developed  for  various  regions  and  contexts 
(Schrodt,  2012).  In  particular,  some  researchers  have  suggested  the  integration  of  event  data  with 
leadership  rhetoric  as  a  way  to  improve  forecasts  of  the  onset  of  intrastate  conflicts  such  as 
insurgencies  (Tikuisis,  Carment,  &  Sarny,  2013). 

It  is  important  to  stress,  however,  that  forecasting  elite  and  organizational  behavior  should 
constitute  a  goal  in  its  own  right  and  not  just  an  auxiliary  input  for  conflict  prediction.  This  will 
stimulate  the  development  of  such  data  at  a  greater  temporal  resolution,  sufficient  to  track  the 
evolution  of  these  groups  and  more  fully  encompass  the  range  of  factors  that  drive  their  behavior. 
Elites  respond  to  the  preferences  of  their  own  organizational  constituents,  the  actions  of  other 
organizations  and  their  leaders,  and  the  sentiment  of  the  broader  population.  Standard  social 
network  analysis  would  focus  on  constructing  ties  only  within  the  set  of  elites  or  between 
organizations,  but  researchers  should  develop  data  that  enables  representation  of  new  multi-level 
networks,  including  the  ties  connecting  leaders  and  the  activist  or  rank-and-file  components  of 
their  organizations  as  well  as  the  ties  among  activists  of  different  organizations. 

The  use  of  both  leadership  tie  indicators  (joint  communiques)  and  tactical  unit  tie  indicators  (joint 
operations)  represents  a  step  toward  better  representations  of  multi-level  networks  that  can  more 
fully  account  for  elite  policy  and  alliance  dynamics.  For  example,  it  is  possible  that  the  standard 
mechanism  of  social  influence,  in  which  actors  with  strong  ties  grow  more  similar  to  each  other 
(Friedkin  &  Johnsen,  2011),  appears  to  be  violated  at  one  level  but  can  be  explained  by  ties 
between  levels.  Such  a  dynamic  explains  the  shift  of  the  IAI  to  a  more  extreme  targeting  policy  in 


Data  processing  for  applications  of  dynamics-based  models  to  forecasting  |  267 


the  August  2007-July  2009  time  period  (Figure  6),  even  though  its  leadership  ties  in  both  periods 
were  exclusively  with  Nationalist  groups  with  more  moderate  targeting  policies,  which  could  have 
been  expected  to  lead  to  a  moderation  of  the  lAI's  targeting  policy.  This  apparent  violation  of  the 
social  influence  mechanism  can  be  explained  by  the  fact  that  the  IAI  was  responding  to  dissent 
from  its  hardline  rank-and-file  constituents  over  its  recent  alliance  with  JAMI  and  HAMI  (Gabbay  & 
Thirkill-Mackelprang,  2010).  The  inclusion  of  the  rank-and-file  level  can  therefore  account  for 
forces  on  elites  that  could  produce  seemingly  anomalous  effects  when  viewed  within  a  single-level 
network  model. 

Representations  of  multi-level  networks  should  also  include  policies  and  ideologies.  This  calls  for 
data  that  can  be  used  to  construct  policy  and  ideological  orientations  of  organizational 
membership  elements.  Social  media  could  provide  one  source  of  data  on  rank-and-file  opinions,  as 
could  analyst  judgments.  Another  approach  would  involve  looking  at  the  rhetoric  in  media 
affiliated  with  elites  whose  audiences  consist  of  their  organizational  members  or  support  bases.  For 
example,  an  analysis  of  the  rhetoric  of  Afghan  government  elites  found  that  the  statements  of  the 
individual  leaders  themselves  were  often  substantially  different,  and  usually  milder,  than  the 
rhetoric  of  their  affiliated  media,  perhaps  indicating  the  more  hardline  beliefs  of  their  bases  of 
support  (Gabbay,  2011).  This  also  highlights  the  value  of  incorporating  into  models  data  that 
connect  elites  with  media  outlets  and  media  outlets  with  their  audiences. 

In  conclusion,  this  chapter  has  highlighted  aspects  of  the  empirical  implementation  of 
computational  models  for  forecasting  purposes.  The  discussion  has  focused  on  dynamics-based 
models,  which  have  the  greatest  potential  to  address  the  most  operationally  challenging  and 
valuable  types  of  forecasting  questions  but  also  make  the  greatest  demands  upon  data  processing. 
Quantitative  variables  and  parameters  for  low-dimensional  models  must  be  coaxed  from  high¬ 
dimensional  and  often  fundamentally  qualitative  data  -  a  problem  compounded  by  the  often 
contested,  chaotic,  and  covert  nature  of  the  environments  and  adversaries  of  most  concern  to 
national  security  intelligence  analysts,  planners,  and  policy  makers.  The  analyst  survey  and  rhetoric 
analysis  methods  described  in  this  chapter  represent  two  different  approaches  for  tackling  this 
problem.  In  practice,  researchers  who  develop  dynamics-based  models  for  forecasting  purposes 
often  place  a  particular  model  formulation  first  and  then  seek  to  collect  and  process  data 
appropriate  for  their  model.  Although  this  section  has  made  a  few  relatively  specific  suggestions  for 
future  research,  a  broader  recommendation  would  be  to  pursue  a  more  integrated  approach  to 
developing  dynamics-based  models  for  forecasting.  Such  an  approach  should  center  on  the 
construction  of  general  quantitative  variables  appropriate  for  a  particular  domain,  such  as  political 
unrest,  insurgencies,  or  proliferation,  which  are  sufficiently  rich  to  enable  the  application  of 
dynamics-based  models  (as  opposed  to  purely  statistical  models),  thereby  encouraging  a  range  of 
approaches  that  can  be  more  readily  evaluated. 
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1.  Introduction 

This  chapter  provides  an  overview  of  computational  sociocultural  models  that  can  support  a 
Forecast  operational  capability.  The  nature  and  basis  of  the  models  that  social  scientists  use  differ 
across  the  various  social  science  disciplines,  and  also  differ  from  the  physical  models  of  traditional 
military  simulation.  Computational  sociocultural  models  require  the  ability  to  represent  and 
manipulate  imprecise  perceptual  and  cognitive  concepts.  This  chapter  provides  an  overview  of  how 
these  models  can  account  for  natural  vagueness  of  human  thought  and  communication,  along  with 
the  dynamics,  complexity,  uncertainty,  variability,  and  unpredictability  of  actions  and  behavior,  in 
order  to  generate  forecasts. 

Figure  1,  inspired  by  a  similar  figure  created  by  Zacharias,  MacMillan,  &  Van  Hemel  (2008), 
illustrates  the  perceived  similarities  among  several  types  of  model  forecasts.  The  models  shown 
span  methods  based  on  mathematics  to  heuristics  with  probability,  complex  dynamics,  soft 
methods  and  networks,  and  judgmental  method  subcategories.  We  selected  these  computational 
sociocultural  models  to  represent  a  broad  set  of  approaches,  but  the  list  is  not  exhaustive.  Since 
2008  (Zacharias  et  al.,  2008),  researchers  have  devoted  considerable  attention  to  analysis  of  online 
and  social  media  data  sources;  where  possible  the  forecast  methods  are  discussed  in  relation  to 
this  type  of  data.  The  approaches  covered  in  this  chapter,  organized  in  9  sections,  include 
probability  forecasts  based  on  (1)  results  from  mixed-method  aggregation,  (2)  emotions  reflected 
in  social  media  text,  and  (3)  causal  Bayesian  methods.  In  addition,  simulation  model  forecasts  are 
reviewed  including  (4)  system  dynamics  models,  (5)  agent-based  models,  (6)  hybrid  models,  and  (7) 
soft  modeling  methods,  which  include  cellular  automata,  evolving  self-organizing  maps,  and 
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artificial  neural  networks.  We  also  describe  forecasts  of  (8)  social  networks  generated  from  text 
analysis,  and  (9)  judgmental  forecasts,  which  include  information  sharing  and  collaboration,  Delphi 
techniques,  and  prediction  markets.  Furthermore,  Chapter  9  in  this  book  discusses  dynamic-based 
models  and  Chapter  11  describes  cellular  automata  in  more  detail. 


Mixed-Method  Aggregation  Forecasting 
Forecasts  of  Emotions  Reflected  in  Social  Media  Text 

Causal  Bayesian  Forecasting 

System  Dynamics 
Agent  Based  Models 
Hybrid  Models 

Cellular  Automata 

Evolving  Self-Organizing  Maps 
Artificial  Neural  Networks 


Forecasts  of  Social  Networks  Generated  from  Text  Analysis 

Information  Sharing  and  Collaboration 
Delphi  Techniques 
Prediction  Markets 


Figure  1.  Forecast  approaches  based  on  mathematics,  models,  and  heuristics. 


Computational  sociocultural  models  used  for  forecasting  |  271 

2.  Probability  Forecasts  Using  Mixed-Method  Aggregation 

The  Defense  Advanced  Research  Projects  Agency's  (DARPA's)  Integrated  Crisis  Early  Warning 
System  (ICEWS)  project  (2007-2009)  demonstrated  the  possibility  of  forecasting  select  events  of 
interest  (EOls)  in  167  countries  with  high  accuracy  by  using  a  mixed  methods  approach  (O'Brien, 
2010).  Under  Assistant  Secretary  of  Defense  for  Research  and  Engineering  ASD(R&E)  funding 
(2010-2013)  through  the  Office  of  Naval  Research,  the  ICEWS  team  conducted  further  research, 
development,  and  operational  testing  and  evaluation,  leading  to  transition  of  ICEWS  capabilities  to 
a  Department  of  Defense  program  of  record. 

ICEWS  provides  commanders  with  relatively  low-cost,  nontraditional  methods  of  Intelligence, 
Surveillance  and  Reconnaissance  (ISR)  that  assist  commanders  in  generating,  monitoring,  and 
assessing  Diplomatic,  Information,  Military,  and  Economic  (DIME)  courses  of  action  to  prevent  or 
mitigate  instability.  ICEWS  forecasts  country  stability  for  five  EOls:  domestic  political  crisis, 
rebellion,  insurgency,  ethnic/religious  violence,  and  international  crisis. 

ICEWS  encompasses  four  primary  sets  of  interoperating  capabilities  (Figure  2):  (1)  iTRACE—  event 
trend  analytics  from  news  media  and  other  sources;  (2)  iCAST— instability  EOI  forecasting;  (3) 
iSENT— sentiment  analysis  from  social  media;  and  (4)  iDIME— instability  prevention  and  mitigation 
via  DIME  actions.  The  system  integrates  these  capabilities  through  a  web-based  portal  and  a  shared 
data  repository.  This  section  focuses  on  the  computational  social  science  models  for  forecasting 
that  make  up  iCAST. 


Figure  2.  ICEWS  Functional  Capabilities. 
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2.1.  iCAST  Functionality 

iCAST  capabilities  include  an  extensible,  mixed-methods  suite  of  computational  social  science 
models  for  forecasting  EOls.  Models  produce  a  monthly  EOI  probability  for  a  given  country  for  six 
months  into  the  future.  The  mixed-methods  approach  integrates  a  number  of  statistical  models  for 
each  EOI  using  a  flexible  model  integration,  data-provisioning,  and  execution  framework  called 
ADvanced  Architecture  for  Modeling  and  Simulation  (ADAMS)  (Wedgwood,  Horiatis,  Siedlecki,  & 
Welsh,  2009)  and  Bayesian  methods  for  aggregating  forecasts  from  those  models  for  each  country 
and  EOI  (Mahoney,  Comstock,  deBlois,  &  Darcy,  2011;  Montgomery,  Ward,  &  Hollenbach,  2011). 
These  methods  generate  an  aggregate  forecast  primarily  by  weighting  each  model's  current 
forecast  by  its  prior  performance  as  compared  to  a  ground  truth  data  set  (GTDS)  based  on  subject 
matter  expert  assessment  of  whether  or  not  an  EOI  occurred  in  a  given  country  in  a  given  month. 
The  iCAST  team  used  the  GTDS  set  covering  2000-2012  to  calibrate  the  statistical  models  and 
validated  them  using  a  split  sample  methodology.  Validity  testing  showed  that  the  aggregate 
forecast  generated  by  the  model  ensemble  outperformed  that  of  any  of  the  individual  models 
(O'Brien,  2010). 

Within  iCAST,  statistical  models  include  logistic  regression  (logit)  models  that  combine  traditional, 
relatively  static,  country-level  indicators  (e.g.,  level  of  democracy,  gross  domestic  product,  ethnic 
fractionalization)  with  more  dynamic  aggregations  of  event  data  coded  from  a  large  corpus  of  news 
stories  dating  back  to  the  year  2000.  These  models  capture  structural  conditions  that  may 
predispose  countries  towards  instability  and  behavioral  factors  such  as  recent  interactions  among 
government,  insurgent,  and  other  types  of  actors.  iCAST  also  includes  hierarchical  statistical  models 
that  exploit  networks  of  relationships  among  countries  such  as  spatial  proximity,  trade  ties,  people 
flows,  etc.  (Ward  et  al.,  2012).  Because  instability  can  "spread"  over  such  networks,  EOls  in  two 
countries  related  by  a  network  may  not  be  truly  independent.  In  addition,  iCAST  contains  zero- 
inflated  negative  binomial  regression  models  that  accommodate  the  frequency  of  non-events  (i.e., 
zero-valued  events).  Furthermore,  the  constituent  statistical  models  often  complemented  one 
another,  with  some  performing  better  on  certain  EOls  and  countries  than  others.  In  later  phases, 
model  performance  improved  from  around  80%  accuracy  (number  of  correct  predictions)  to  over 
95%,  with  less  than  5%  false  positives. 

2.2.  Analyses  Supported 

iCAST  increases  users'  awareness  of  human,  social,  cultural  and  behavioral  (HSCB)  characteristics 
describing  the  historical,  current,  and  projected  sociocultural  situation  in  a  country.  The  forecasts 
themselves  are  essentially  heuristics  that  guide  users  as  they  search  the  massive  space  of  HSCB 
data. 

At  the  most  basic  level,  iCAST  enables  users  to  view  forecasts  for  different  countries  and/or  EOls  to 
explore  such  questions  as,  "What  is  the  anticipated  probability  of  an  insurgency  in  Country  X?"  The 
distribution  of  probabilities  for  a  particular  <EOI,  country>  pair  can  help  users  to  identify  forecasts 
that  merit  deeper  investigation.  In  Figure  3,  forecasted  probabilities  are  shown  as  an  orange  line. 
The  plot  area  for  historical  data  points  are  shaded  either  black  (event  occurred)  or  white  (event  did 
not  occur).  For  forecasts,  the  plot  area  is  shaded  depending  on  forecasted  probability  where  red 
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are  high-probability  events  (>=67%),  yellow  are  medium-probability  events  (>=33%  and  <67%),  and 
green  are  low-probability  events  (<33%). 


Insurgency 

[Historical  and  Aggregated  Forecasts) 


Figure  3.  iCAST  showing  aggregated  forecasts  generated  for  country-level  insurgency. 


Once  the  user  has  chosen  a  forecast  for  further  examination,  iCAST  decomposes  the  aggregate 
forecast  to  permit  analysis  of  individual  model  forecasts.  This  drilldown  informs  the  user  how  each 
individual  forecast  contributes  to  the  aggregate  (Figure  4).  The  table  at  the  top  of  Figure  4  shows 
the  models  contributing  to  the  aggregation  and  the  aggregate  model  itself.  The  table  also  lists 
individual  model  forecasts  and  impact  for  each  model.  In  this  example,  Model  4  has  the  greatest 
impact  on  the  aggregate  forecast.  The  bottom  of  this  view  shows  the  accuracy  (blue),  precision 
(red),  and  recall  (green)  analysis  for  each  model. 
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Figure  4.  "Aggregated  Forecast  Drilldown"  view  for  a  specified  country,  EOI,  and  time  (i.e.,  month 
and  year). 


Aggregated  Forecast  Drilldown  for  Insurgency  for  January  2013 


Computational  sociocultural  models  used  for  forecasting  |  275 

When  the  user  identifies  a  specific  model  for  further  analysis,  iCAST  enables  a  more  detailed 
exploration  with  respect  to  the  underlying  data  parameters  informing  the  model's  forecasts.  For 
example,  the  user  may  ask,  "What  key  factors  contribute  to  the  likelihood  of  an  insurgency's 
occurring,  or  not,  in  a  country?"  The  visualizations  show  the  user  the  variables  used  by  a  model, 
how  those  variables  have  changed  over  time  with  respect  to  occurrence  of  the  EOI,  and  which 
variables  have  impact  on  the  current  forecast  calculation  (Figure  5). 

iCAST  analyzes  the  impact  of  each  variable  on  the  model's  forecast  by  performing  sensitivity 
sweeps  in  which  a  single  variable  is  varied  discretely  over  a  historically  observed  range  of  values 
while  all  other  variables  are  left  at  their  currently  observed  levels.  If  fluctuations  in  a  specific 
variable  alter  the  model's  forecasted  probability  (<=0.2%),  the  variable  name  is  shaded  blue.  If  it 
alters  the  probability  slightly  (>0%  change,  but  <2%  change),  the  variable  is  shaded  yellow.  If 
changes  in  the  variable  value  produce  no  change,  the  variable  is  shaded  gray. 

To  help  users  better  understand  the  variables  used  by  the  models,  iCAST  permits  drill  down  into  a 
view  of  the  variable's  performance  over  time.  From  this  view,  the  user  can  identify  a  particular 
value  of  the  variable  to  investigate  further.  Furthermore,  iCAST  enables  the  user  to  access  the  news 
stories  from  which  the  model  extracted  events  that  contributed  to  the  calculation  of  a  particular 
variable. 


Government  toward  nongovernment  actors  -  Fight  count 

Population  total 

State  Department  Measure  from  Political  Terror  Scale 

Amount  of  democracy  (Combined  Polity  Score) 

Dissidents  toward  nondissident  actors  -  High  hostility  total 

GDP  (constant  2000  US$) 

Institutionalized  Autocracy 

Separatist  High  Hostility  Event  Counts  Both 

All  actors  toward  all  actors  -  High  hostility  count 

Figure  5.  Screenshot  of  the  variables  list  of  a  specific  model  presented  to  better  understand  what 
factors  contribute  to  the  model's  analysis. 


2.3.  Modeling  Considerations 

Operators  are  particularly  interested  in  the  onset  and  cessation  of  EOls  (e.g.,  when  a  rebellion 
starts  or  ends)— a  subset  of  the  already  sparse  set  of  occurrences  of  the  EOls,  and  therefore  hard 
to  forecast  with  accuracy.  Operators  also  prefer  statistical  models  that  use  actionable  indicators 
(i.e.,  variables  that  are  able  to  be  influences  by  DIME  actions)  grounded  in  social  science  theory  as 
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input.  The  relatively  rare  occurrence  of  the  focus  EOls  in  the  GTDS  makes  it  difficult  to  identify 
actionable  indicators. 

In  addition,  operators  prefer  models  that  accurately  predict  EOls  in  multiple  countries  over  those 
focused  on  just  one  country.  This  led  the  iCAST  team  to  develop  mixed  effect  models  in  which  the 
coefficients  of  some  variables  can  vary  by  country.  The  team  also  created  models  for  sets  of 
countries  grouped  by  polity  type  (e.g.,  democracy,  autocracy,  etc.).  These  techniques  improved 
model  performance  and  actionability. 

Models  must  foster  operators'  trust  in  the  results  by  providing  actionable  detail  from  the  forecasts. 
In  iCAST,  the  complexity  of  using  an  aggregated  forecast  made  up  of  multiple,  heterogeneous 
models  complicates  transparency.  Presenting  the  aggregated  model  results  in  a  useful  way 
required  three  interrelated  capabilities:  (1)  understanding  what  data  the  models  use  (including 
how  model  inputs  are  calculated),  (2)  understanding  the  social  science  theory  underlying  the 
models  and  their  inputs  and  outputs,  and  (3)  presenting  the  aggregated  and  individual  forecast 
results  to  support  traceability  in  a  clear  and  easily  navigable  user  interface. 

To  this  end,  the  iCAST  team  developed  a  framework  for  model  forecast  transparency  (Wedgwood, 
Ruvinsky,  &  Siedlecki,  2012)  that  reveals  the  driving  variables  of  each  model,  shows  how  those 
variables  performed  under  historical  conditions,  and  supports  exploration  into  how  changing 
conditions  may  affect  forecasts.  Interactive  visualizations,  such  as  those  shown  in  Figure  6, 
generally  provide  that  transparency  for  end  users  (operators,  analysts)  and  model  developers. 
These  views  include  heat  maps  showing  forecasted  EOls  and  details  on  aggregate  forecasts  (from 
the  aggregation  methods)  and  on  individual  model  forecasts,  including  which  input  variables  (and 
values)  had  what  impact  on  each  model's  forecast. 

iCAST  supports  a  "what-if"  capability  for  exploring  how  fluctuations  in  model  inputs  affect  the 
forecasted  probability  generated  by  the  individual,  constituent  models  as  well  as  the  aggregate 
model.  Identifying  the  variables  at  play  in  an  individual  model  forecast  can  lead  users  to  inquire 
how  changing  those  variables  might  affect  the  probability  of  an  EOI.  iCAST  lets  users  explore  data 
parameter  sensitivity  within  a  specific  model  forecast  (Figure  6)  to  see  how  changes  in  a  variable 
affects  the  individual  model's  forecast.  iCAST  also  supports  exploring  how  the  impact  on  the 
individual  model  forecast  then  influences  the  aggregate  model's  assessment. 

Some  of  these  visualizations  also  permit  "what  if"  exploration  (Figure  7):  allowing  users  to  alter  the 
values  of  model  input  variables  and  observe  the  resulting  impact.  This  enables  users  to  gain  an 
intuitive  feel  for  the  relationships  among  the  input  variables  and  model  output.  Users  can  also 
simulate  DIME  actions  and  their  impact  on  EOI  probabilities  by  altering  these  "levers."  Other 
transparency  views  display  a  model's  pedigree,  track  record  of  accurate  predictions,  etc. 
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Figure  6.  Potential  impact  of  model  input  variables  on  probability  of  an  International  Crisis  in  a 
specified  country  and  time  period.  This  view  shows  that  the  variable  ''Government  human  rights 
violations"  (GHRV)  can  impact  the  current  EOI  forecasted  of  0.86. 
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Figure  7.  iCAST  visualization  allows  users  to  investigate  "what-if"  scenarios.  This  view  shows  the 
current  value  of  all  model  variables  and  forecasted  probability  in  orange.  The  user  can  manipulate 
levels  of  each  variable  to  see  how  changes  in  variable  values  affect  the  EOI  probability  shown  in 
blue  (variable  changes  increase  the  forecast  from  0.5  to  0.8). 
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An  ontology  of  epistemological  elements  that  captures  model  design  from  the  underlying  theory 
has  been  developed  and  operationalized  with  specific  data  sources  to  improve  model  verification 
and  validation  (Ruvinsky,  Wedgwood,  &  Welsh,  2012).  iCAST  has  used  such  an  ontology  to  extend 
information  and  understanding  regarding  model  designed  and  performance. 


3.  Forecasts  of  Emotions  Reflected  in  Social  Media  Text 

Researchers  have  developed  new  methods  to  characterize  the  dynamics  of  emotions  in  large 
populations  as  captured  in  Twitter  data  (Servi,  2013;  Servi  &  Elson,  2012).  To  cull  the  right  subset  of 
Twitter  data,  researchers  must  first  specify  the  dates,  hashtags,  key  words,  and/or  geographical 
region(s)  of  interest.  Second,  as  shown  in  Figure  8,  they  must  convert  the  words  of  Twitter 
messages  into  numerical  data  as  shown  on  the  y-axis.  This  can  be  done  using  a  bag-of-words 
approach.  Here  the  researcher  formulates  a  collection  of  words  -  "the  bag"  -  associated  with  an 
emotion  of  interest  and  compares  them  to  the  actual  words  in  the  tweets  to  compute  a  score  for 
each  day  and  for  each  emotion.  This  can  be  done  by  associating  an  intensity  of  the  emotion, 
sometimes  referred  to  as  a  "valence,"  with  each  word  (e.g.,  Bradley  &  Lang,  1999;  Dodds  & 
Danforth,  2010)  or  by  treating  each  word  with  the  same  intensity.  Linguistic  Inquiry  and  Word 
Count  (LIWC),  a  bag-of-words  approach  that  has  an  extensive  scientific  pedigree  and  has  been 
validated  across  a  number  of  studies  (e.g.,  Pennebaker,  Booth,  &  Francis,  2007;  Pennebaker, 
Chung,  Ireland,  Gonzales,  &  Booth,  2007;  Pennebaker  &  Francis,  1996),  was  used  to  generate 
Figure  8. 
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Figure  8.  Normalized  percentages  of  LIWC  swear  and  sadness  words  in  weekly  aggregates  of 
tweets. 


While  bag-of-words  approaches  rely  on  word  distributions  and  can  also  use  co-occurrence 
associations  to  tease  apart  the  possible  meaning  of  words  in  text,  a  different  approach  seeks  to 
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extract  meaning  by  considering  the  role  of  words  within  the  structure  of  the  phrase  or  sentence. 
The  annotated  Multi-Perspective  Question  Answering  data  set  (Wiebe,  Wilson,  &  Cardie,  2005) 
captures  details  about  the  sentiments  and  opinions  expressed  by  or  attributed  to  opinion  holders, 
the  intensity  of  those  sentiments,  and  the  propositions  or  entities  toward  which  these  sentiments 
are  directed.  This  approach  was  used  to  annotate  each  word,  phrase,  and  sentence  within  some 
600  news  and  editorial  articles,  enabling  words-in-context  approaches  to  sentiment  analysis  to  be 
trained  and  evaluated  against  this  data  set  (Day,  Boiney,  Ubaldino,  &  Brown,  2012;  Kim  &  Hovy, 
2006;  Wilson,  2008).  Pang  and  Lee  (2008)  provide  an  overview  of  Natural  Language  Processing 
approaches  to  sentiment  analysis. 

The  third  step,  illustrated  in  Figure  9,  requires  algorithmically  filtering  some  of  the  measurement 
error.  This  neccesates  a  model  of  the  dynamics  of  emotions.  Servi  and  Elson  (2012)  postulate  that 
emotions  follow  piecewise  linear  dynamics  until  an  event  external  to  the  system  creates  a 
discontinuous  change  -  in  essence  an  emotional  "reset"  -  after  which  the  dynamic  resumes.  The 
authors  use  the  term  "breakpoint"  to  describe  this  discontinuous  reset.  Researchers  have 
developed  algorithms  for  such  a  filtering  approach,  as  well  as  a  number  of  generalizations  of  that 
approach  (Servi,  2013). 


Figure  9.  Breakpoint  analysis  of  normalized  percentages  of  LIWC  swear  and  sadness  words  in  daily 
aggregates  of  tweets,  using  a  linear  model. 
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Forecasting  efforts  seek  to  predict  future  events  using  input  along  the  lines  of  Figure  9  in  a 
scientifically  rigorous  manner.  Predicting  future  UWC  data  scores  represents  a  first  step  in  this 
direction.  This  requires  that  predictions  be  compared  to  a  baseline.  The  natural  baseline  is  to 
assume  that  future  emotional  intensities  would  be  the  same  as  those  in  the  past.  More  formally, 
the  baseline  is: 

Rule  1:  Use  the  last  dot.  Forecast  the  value  of  a  UWC  indicator  without  using  breakpoint  analysis  by 
assuming  that  the  last  data  point  is  the  best  estimate  for  the  future.  If  this  rule  outperforms  the 
others,  it  will  mean  that  breakpoint  analysis  is  not  necessary. 

This  rule  was  compared  with  two  other  rules  (Servi  &  Elson,  2012): 

Rule  2:  Project  ahead.  Perform  a  breakpoint  analysis  with  the  linear  model  and  use  that  analysis  to 
make  forecasts  by  projecting  ahead  in  time,  starting  from  the  most  recent  breakpoint.  That  is, 
extend  the  most  recently  derived  trend  line  (continuing  along  its  current  slope)  and  use  that  line  to 
determine  the  predicted  value.  This  is  the  most  natural  approach.  However,  as  indicated  below,  it 
may  demonstrate  a  comparative  advantage  only  if  extensive  data  exist  in  each  individual 
breakpoint  region  and  if  the  trend  lines  have  large  slopes. 

Rule  3:  Use  the  last  estimate.  Perform  a  breakpoint  analysis  with  the  linear  model  and  use  that 
analysis  to  forecast  that  future  values  will  be  the  same  as  the  last  estimated  data  point  in  time. 
Extend  the  most  recently  derived  trend  line,  but  with  a  horizontal  slope,  rather  than  continuing 
along  the  current  slope  of  the  line.  While  this  may  be  less  accurate  and  less  intuitive  than  Rule  2,  if 
data  are  plentiful,  this  more  conservative  and  robust  rule  will  perform  well  for  a  broader  range  of 
data  availability  and  slopes  of  the  trend  lines. 

Table  1  illustrates  the  result  of  a  typical  study  (Servi  &  Elson,  2012)  that  represents  analysis  using 
the  two  datasets  (swear  and  sadness  words)  illustrated  in  Figure  8  and  Figure  9.  After  a  warm-up 
period  of  100  days  the  researchers  made  predictions  1  day,  8  days,  and  15  days  into  the  future  and 
compared  the  prediction  with  the  actual  data.  While  the  results  call  for  more  validation,  the  key 
finding  depicted  shows  that  for  these  data  Rule  3  consistently  outperformed  both  the  baseline  Rule 
1  and  Rule  2.  In  addition,  Rule  2  outperformed  the  baseline  Rule  1  when  applied  against  the 
normalized  swear  word  data  but  fell  short  when  applied  against  the  normalized  sadness  data  (not 
normalized  to  tweet  volume  but  to  each  other).  Table  2  lists  the  sum  of  the  square  of  the 
difference  between  the  percentage  of  LWIC  words  predicted  compared  with  the  actual  number. 
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Table  1.  Sum  of  square  error  analysis  in  predicting  normalized  UWC  swear  words  for  daily  data 


1  day 

8  days 

15  days 

Baseline  rule 

42 

50 

52 

Project  ahead  rule 

37 

43 

49 

Use  last  estimate  rule 

34 

35 

35 

Improvement 

19% 

35% 

35% 

Table  2.  Sum  of  square  error  analysis  in  predicting  normalized  UWC  sadness  words  for  daily  data 


1  day 

8  days 

15  days 

Baseline  rule 

17 

25 

29 

Project  ahead  rule 

25 

49 

69 

Use  last  estimate  rule 

17 

22 

22 

Improvement 

0% 

12% 

24% 

Rule  2  might  have  performed  relatively  poorly  because  the  linear  rate  of  change  of  UWC  scores 
with  respect  to  time  is  imperfectly  estimated.  Hence,  a  large  component  of  the  prediction  error  of 
Rule  2  may  result  from  the  error  in  estimating  the  rate  of  change  of  the  UWC  scores,  and  in  some 
cases  that  misestimation  may  dominate  it.  Therefore,  Rule  2  might  perform  better  as  the  number 
of  data  points  in  the  breakpoint  region  increases,  the  rate  of  change  in  the  UWC  scores  increases  in 
absolute  value,  and/or  the  error  in  the  UWC  estimates  decreases.  The  structural  nature  of  this 
source  of  error  has  been  recognized  in  the  literature  (Servi,  2013;  Tibshirani,  1996). 

Research  has  just  begun  to  examine  another,  more  fundamental  challenge  to  forecasting  through 
modeling  of  emotions:  the  dynamics  of  emotions  may  be  inherently  nonlinear.  This  would  have 
profound  implications,  such  as: 

•  The  accuracy  of  linear  forecasting  models,  such  as  those  described  above,  may  be  self¬ 
limited.  The  limitation  could  be  quantified  by  proper  examination  of  the  nonlinear 
dynamics. 
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•  The  accuracy  of  forecasting  could  be  improved  beyond  the  best  linear  forecasts  by 
estimating  the  characteristics  of  nonlinear  dynamics  data  and  using  them  to  create  new 
forecast  models. 

•  Exploring  characteristics  found  only  in  nonlinear  dynamical  systems,  such  as  the 
phenomenon  of  Critical  Slow  Down  (CSD)  before  abrupt  changes  observed  in  many 
domains  with  nonlinear  dynamics,  could  lead  to  a  different  class  of  forecast  models  (Drake 
&  Griffin,  2010;  Scheffer  et  al.,  2009;  Slater,  2012). 


4.  Causal  Bayesian  Forecasts 

A  forecast  is  a  statement  of  the  probability  that  a  proposition  will  hold  at  a  given  point  in  the 
future.2  Researchers  can  use  statistical  models  for  forecasting  provided  they  have  assembled  a 
large  data  base  of  indicators  containing  positive  and  negative  examples  of  the  proposition  of 
interest.  For  example,  a  statistical  forecasting  model  might  outperform  random  guessing  at 
computing  the  probability  it  will  rain  tomorrow  based  on  today's  temperature,  relative  humidity, 
cloud  cover,  and  so  on.  In  situations  characterized  by  known  trends  related  to  the  question  of 
interest  and  the  availability  of  reliable  time  series  data,  researchers  can  build  auto-regression 
forecasting  models  such  as  those  used  to  forecast  macroeconomic  conditions  (Granger,  1969).  The 
U.S.  Commerce  Department  forecasts  the  coming  quarter's  Gross  Domestic  Product  based  on  the 
prior  quarters'  housing  prices,  unemployment  rates,  etc. 

However,  statistical  forecasting  cannot  predict  rare  events  (e.g.,  the  probability  that  al-Qaida  will 
develop  an  improvised  nuclear  weapon  by  2015)  because  the  required  large  database  of  prior 
examples  does  not  exist.  Further,  statistical  methods  cannot  identify  or  predict  the  impact  of 
interventions  (e.g.,  how  that  probability  would  change  if  U.S.  forces  expelled  al-Qaida  from  safe 
havens  in  Africa)  unless  the  model  contains  causal  factors.  For  example,  the  National  Weather 
Service  does  use  statistical  models  based  on  causal  factors  such  as  wind  patterns  and  barometric 
pressure  to  forecast  events  such  as  the  point  of  landfall  of  a  hurricane. 

Predicting  rare  events  and  the  impact  of  interventions  requires  modeling  of  causal  chains  of  actions 
and  events.  Researchers  who  study  causality  use  "neuron  diagrams"  to  represent  cause-effect 
relationships.  For  example,  Figure  10  depicts  a  model  of  possible  causes  of  a  carpet  becoming 
soaked  with  water  from  an  aquarium. 


1  This  exposition  uses  the  terms  "forecast"  and  "prediction"  interchangeably. 
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Pam  aims  Pam  throws 


Figure  10.  Causal  Network. 


Causal  models  result  from  enumerating  possible  sequences  of  actions  that  might  lead  to  various 
outcomes.  Because  planning  consists  of  deliberating  to  construct  a  chain  of  actions  that  would 
bring  about  a  desired  state,  a  plan  is  a  kind  of  causal  model:  actions  cause  effects  (usually  with 
some  uncertainty)  and  some  of  the  anticipated  effects  of  actions  either  constitute  the  goal  of  the 
plan  or  establish  preconditions  for  subsequent  actions  that  eventually  bring  about  the  goal.  Plans 
typically  are  represented  by  Kripke  networks  (Kripke,  1963),  which  can  be  imagined  as  a  neuron 
diagram  having  choice  points  and  branches  leading  to  various  outcome  states,  including  the  goal 
state  and  various  ways  the  plan  might  fail  to  reach  the  goal  state. 

In  the  absence  of  a  deterministic  cause-effect  relationship  (e.g.,  throwing  a  rock  at  an  aquarium 
does  not  always  cause  the  glass  to  shatter;  stopping  rock  throwing  does  not  guarantee  the  carpet 
will  not  be  soaked),  neuron  diagrams  (or  plan  networks)  can  be  augmented  with  state  transition 
probabilities.  This  transforms  a  causal  network  into  a  Causal  Bayesian  network  (CBN)  (Pearl,  2000). 
When  nodes  represent  random  variables  and  edges  represent  conditional  dependencies  among  the 
variables,  the  directed  acyclic  graph  is  called  a  Bayesian  Belief  Network  (BBN).  As  illustrated  in 
Figure  11,  each  node  that  has  at  least  one  parent  node  has  an  associated  conditional  probability 
table  in  which  rows  correspond  to  a  combination  of  parent  node  states.  The  values  in  each  row 
constitute  the  conditional  probability  distribution  over  the  node's  states  given  an  enumerated 
combination  of  parent  node  states.  A  node  with  no  parents  means  the  random  variable  is 
independent  of  the  other  variables  in  the  network  (e.g.,  node  C  -  cloudy).  Parentless  nodes  contain 
the  unconditional  prior  probability  distribution  over  that  node's  states. 
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Figure  11.  Bayesian  Belief  Network. 


Such  graphs  are  called  BBNs  because  they  use  Bayes'  rule  to  compute  the  belief  of  every  state  of 
every  node  in  the  network.  In  this  example,  probability  (P)  is  the  belief  that  a  state  holds,  whereas 
a  statistical  model  would  compute  the  frequency  with  which  the  outcome  state  occurred  (e.g.,  it 
rained)  in  relationship  to  the  number  of  times  the  given  combination  of  factor  values  was  observed 
(e.g.,  the  weather  was  cloudy  and  hot  on  the  previous  day).  These  beliefs  are  updated  upon 
observation  of  any  combination  of  states  of  the  variables  represented  by  the  network.  For 
example,  in  the  BBN  in  Figure  11,  if  the  modeler  knows  that  the  grass  was  not  wet  on  a  given  day, 
then  the  probability  that  it  was  cloudy  on  that  day  would  decrease  from  0.5  to  0.36,  the  belief  that 
the  sprinkler  was  on  from  0.3  to  0.06,  and  the  belief  that  it  rained  from  0.5  to  0.12. 

Importantly,  providing  evidence  in  any  node  in  a  BBN  updates  all  the  other  state  beliefs.  Thus, 
BBNs  can  be  used  for  evidential  reasoning  (e.g.,  dry  grass  is  evidence  that  it  was  not  cloudy,  that  it 
did  not  rain,  and  that  the  sprinkler  was  not  used)  and  for  prediction  (e.g.,  looking  out  of  a  window 
and  observing  a  cloudy  -  or  sunny  -  sky  allows  individuals  to  assess  the  probability  that  they  will 
encounter  a  running  sprinkler  or  wet  grass  later  in  the  day). 

A  BBN  becomes  a  causal  Bayesian  model  when  some  of  the  nodes  represent  actions,  their  effects, 
or  preconditions,  and  the  links  represent  causal  relationships  such  as  those  in  a  neuron  diagram. 
One  can  create  a  causal  model  using  subject  matter  expertise.  Adding  the  probability  that  each 
action  will  succeed  given  the  possible  states  in  which  the  action  is  selected  for  execution  turns  that 
structure  into  a  Causal  Bayesian  Network  (CBN).  CBNs  can  be  used  to  analyze  courses  of  action 
(COAs)  -  another  term  for  plans.  Thus,  CBNs  that  represent  plans  can  be  used  to  forecast  the 
possible  outcomes  of  the  plans.  COA  analysis  involves  representing  a  plan  as  a  CBN  and  using  the 
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causal  structure  of  the  CBN  to  identify  the  outcome  of  interventions,  such  as  stopping  key  actions 
by  negating  their  preconditions.  A  CBN  representation  can  help  to  predict  if  an  agent  will  follow  a 
particular  plan  by  using  plan  recognition  technology  to  generate  possible  explanations  in  the  form 
of  plans  derived  from  observed  evidence.  This  becomes  particularly  valuable  in  situations  where 
adversaries  actively  attempt  to  deny  information  to  observers  (Burke,  Coplon,  Elsaesser,  Hitzelman, 
Mireles,  Rothleder,  &  Tanner,  2005). 


5.  System  Dynamics  Forecasts 

System  dynamics  is  a  simulation  approach  used  to  simulate  the  behavior  of  a  system  over  time  as  a 
consequence  of  its  top  down  design  and  causal  relationships.  It  has  proven  useful  in  modeling 
organizational  behaviors  driven  by  complex  causal  structures.  Since  its  first  application  to  modeling 
production  and  distribution  dynamics  in  industry  (Forrester,  1958),  researchers  have  applied 
system  dynamics  to  simulate  organizational  behavior,  both  to  create  generalized  theory 
(Repenning  &  Sterman,  2001)  and  to  model  specific  systems  for  forecasting,  policy  development, 
and  design  recommendations  (Lyneis,  2000).  In  addition,  system  dynamics  has  been  used  to  study 
such  areas  as  strategy  (Fowler,  2003),  supply  chains  (Scheiritz  &  GroBler,  2003),  product 
development  (Ford  &  Sterman,  1998)  and  business  processes  (Ashayeri,  Keij,  &  Broker,  1998). 

The  system  dynamics  modeling  approach  can  serve  as  a  hands-on  organizational  learning  tool  to 
help  managers  discover  and  understand  the  structures  underlying  the  dynamics  of  their 
organizations.  ''Flight  simulator"  models  of  organizations  are  often  used  to  help  teach  systems 
thinking  by  building  up  simulated  experiences  of  complex  systems  (Fowler,  2003;  Sterman  2000). 
Vennix  (1998)  has  developed  a  corresponding  group  facilitation  approach  to  simultaneously  allow 
group  participants  to  create  qualitative  "causal  loop  diagrams"  and  quickly  test  the  implications  of 
the  feedback  logic  of  those  diagrams  with  small  system  dynamics  simulations.  Modelers  can  then 
build  and  validate  more  advanced  system  dynamics  simulations  that  use  this  information  to  make 
forecasts  or  design  strategies  to  create  new  models  of  behavior  in  real-world  systems. 

All  but  the  most  complex  system  dynamics  simulations  run  in  near-real  time,  enabling  users  to 
perform  "what  if"  analyses  rapidly  and  easily.  The  causal  structure  of  system  dynamics  models  also 
allows  decision  makers  to  quickly  identify  the  key  policy  levers  that  control  system  behavior.  The  C- 
ROADS  model  of  global  climate  change  (Sterman  et  al.,  2012)  presents  one  example  of  such  a 
model. 

System  dynamics  uses  simple  diagrams  to  represent  the  causal  dependencies  in  a  system.  The 
diagrams  present  processes  in  terms  of  "stocks"  (e.g.,  people,  materials,  knowledge,  money), 
materially  conserved  "flows"  between  the  stocks,  and  causal  information  about  stocks  and 
parameters  that  determine  these  flows.  Users  populate  these  "stock  and  flow  diagrams"  with  data 
to  simulate  the  behavior  of  a  system,  as  illustrated  in  Figure  12,  a  simple  model  of  disease 
dynamics.  Flows  (double-lined  arrows)  and  information  arrows  (single-lined  arrows)  connect 
stocks,  variables,  and  constants.  The  polarity  of  these  causal  connections  is  shown  as  either  a  "+," 
to  indicate  a  positive  relationship,  or  a  to  indicate  a  negative  relationship. 
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Figure  12.  System  dynamics  depiction  of  the  Susceptible,  Infectious,  Recovered  (SIR)  model  of 
epidemiology. 


The  behavior  of  system  dynamics  models  is  driven  mainly  by  feedback  loops.  In  Figure  12,  a 
powerful  reinforcing  "R"  feedback  loop  allows  the  system  to  rapidly  increase  the  Infection  Rate 
when  the  stock  of  Susceptible  Population  is  large.  This  occurs  because  the  larger  the  Infectious 
Population,  the  more  likely  its  members  are  to  have  contact  with  the  Susceptible  Population. 
Similarly,  two  balancing  "B"  feedback  loops  limit  and  ultimately  end  a  wave  of  infection.  If  the 
Susceptible  Population  decreases,  it  contains  fewer  people  to  infect.  If  the  Recovered  Population 
increases,  it  means  that  more  people  are  no  longer  susceptible  to  infection. 

The  model  shown  in  Figure  12  is  a  very  simple  one.  Complex  feedback  systems  can  be  modeled  by 
using  more  variables  and  linking  them  to  form  more  feedback  loops.  No  matter  how  complex, 
however,  the  behaviors  of  these  models  can  be  broken  down  into  classic  dynamic  patterns  of 
behavior,  such  as  exponential  growth  and  decay,  S-shaped  growth,  growth  and  collapse,  and 
oscillatory  behavior. 

Because  system  dynamics  fundamentally  relies  on  modeling  the  causal  dependencies  in  a  system, 
modelers  must  have  a  firm  mathematical  understanding  of  each  of  these  relationships,  as  well  as 
supporting  data  and  metrics  if  the  system  is  to  be  simulated.  Flowever,  quantitatively  capturing 
these  relationships  in  system  dynamics  models  of  "soft"  processes,  such  as  strategy,  presents 
considerable  challenges.  System  dynamics  models  force  modelers  to  explicitly  link  data  and  causal 
structures  throughout  the  system.  This  can  also  aid  in  uncovering  problems  related  to  data 
collection  and  model  structure. 

Calibrating  models  with  historical  data  and/or  running  a  sensitivity  analysis  (e.g.,  Monte  Carlo)  may 
help  alleviate  or  highlight  concerns  with  both  input  data  and  model  structures  of  abstract  concepts. 
By  conducting  a  sensitivity  analysis,  a  modeler  can  determine  whether  a  system  has  "tipping 
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points"  or  extreme  sensitivity  that  must  be  discovered  and  managed.  A  Monte  Carlo  analysis 
generates  a  range  of  estimated  future  values  for  a  variable,  rather  than  point-in-time  values  for 
individual  model  runs.  The  output  of  this  analysis  can  be  visualized  as  a  "hurricane  chart"  that 
displays  a  confidence  range  in  model  output  over  time,  as  shown  in  Figure  13. 


Figure  13.  Example  of  a  Monte  Carlo  analysis  showing  the  range  of  values  of  a  stock  in  a  system 
dynamics  model.  Uncertainty,  shown  in  the  shaded  area,  grows  over  time.  The  orange  line  is  one  of 
the  hundreds  of  runs  used  to  compute  the  area  of  uncertainty. 


5.1.  System  Dynamics  for  Forecasting 

The  literature  describes  many  examples  of  using  system  dynamics  for  forecasting,  especially  for 
various  forms  of  demand  (Niu  &  Gillard,  1994;  Tharmmaphornphilas,  Loharsiriwat,  &  Vannasetta, 
2012;  Najjar,  2013).  Tharmmaphornphilas,  Loharsiriwat,  and  Vannasetta  (2012)  compared  their 
results  to  that  of  a  Box  Jenkins  (1970)  forecast  and  attributed  their  improved  prediction  results  to 
the  inclusion  of  qualitative  influences  ignored  by  other  methods  and  of  interactions  and  feedback 
in  the  system  that  are  not  apparent  from  an  analysis  of  the  data  alone. 

System  dynamics  is  best  used  for  forecasting  models  where  the  causal  structure  of  the  system, 
rather  than  its  initial  parametric  conditions,  drives  system  behavior  ("structure  drives  behavior").  If 
researchers  have  no  solid  hypothesis  to  support  an  underlying  structure  that  drives  a  system's 
dynamics,  data-driven  approaches  such  as  regression  are  more  appropriate.  System  dynamics 
models  are  generally  better  suited  to  providing  strategic  insight  into  problems  and  for  comparing 
alternative  policies  than  to  making  point  predictions  subject  to  noise  and  externalities. 

5.2.  System  Dynamics  Challenges 

To  perform  effectively,  system  dynamics  models  must  incorporate  all  the  feedback  loops  and 
variables  relevant  to  understanding  the  problem.  Models  of  management  systems,  for  example, 
must  capture  concepts  such  as  "schedule  pressure,"  which  can  lead  to  lower  quality  and  then  an 
increase  in  re-work,  thereby  slowing  production.  Because  of  the  focus  on  causal  structure,  missing 
such  a  causal  loop  can  greatly  alter  the  model's  behavior.  System  dynamics  modeling  only  produces 
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useful  results  when  the  assumption  of  aggregation  is  plausible.  If  the  stocks  consist  of 
heterogeneous  individuals  or  resources,  and  if  this  heterogeneity  is  important  to  system  behavior, 
a  system  dynamics  model  may  provide  misleading  results.  The  reliance  on  aggregate 
representations  of  entities  in  system  dynamics  makes  it  easy  to  implement  models  of  high-level 
system  structure,  but  complicates  low-level  modeling.  System  dynamics  modelers  have  responded 
to  this  weakness  by  increasing  the  number  of  stocks  or  states  used  in  their  models,  but  this  makes 
models  more  complex,  harder  to  debug,  and  harder  to  explain  to  users. 


6.  Agent-Based  Model  Forecasts 

Fundamentally,  an  Agent-Based  Model  (ABM)  is  a  simulation,  and,  like  all  simulations,  can  take 
many  forms.  Typically,  modelers  create  these  simulations  using  a  modern  computer  programming 
language  such  as  Java,  but  they  could  also  create  an  ABM  using  ball  bearings,  animals,  or  even 
human  actors. 

ABMs  differ  from  many  other  simulations  in  the  way  they  focus  on  the  system  studied:  they 
explicitly  represent  all  of  the  relevant  entities  and  their  interactions  within  the  system.  This  makes 
them  especially  useful  for  representing  and  studying  human  systems,  which  may  involve  many 
different  kinds  of  actors  with  a  large  amount  of  heterogeneity. 

At  the  very  least,  ABMs  consist  of  agents  (discrete  purposive  entities,  such  as  humans),  a 
meaningful  environment  (typically  a  geospatial  environment,  but  sometimes  a  more  abstract 
space),  and  a  set  of  rules  that  define  interactions  (among  agents  and,  potentially,  among  agents 
and  the  environment).  Often  the  interaction  rules  contain  random  components,  meaning  that  two 
agents  may  interact,  but  that  interaction  is  not  guaranteed.  This,  in  turn,  means  that  each  run  of  an 
ABM  is  unique.  Therefore,  to  understand  the  results  users  must  run  the  ABM  many  times,  varying 
the  random  conditions.  General  information  about  ABMs  and  their  use  can  be  found  in:  Artificial 
War  (llachinski,  2004),  Agent-Based  and  Individual-Based  Modeling  (Railsback  &  Grimm,  2012), 
Simulation  for  the  Social  Scientist  (Gilbert  &  Troitzsch,  2005),  and  Growing  Artificial  Societies 
(Epstein  &  Axtell,  1996). 

6.1.  Advantages  of  Agent-Based  Models 

For  certain  applications,  ABMs  have  advantages  over  traditional  methods  used  to  study  human 
systems.  For  example,  traditional  systems  dynamics  and  differential  equations  disregard  space, 
meaning  that  all  the  humans  represented  in  these  systems  are  at  the  same  place  at  the  same  time 
and  everyone  interacts  continuously  with  everyone  else.  Furthermore,  only  very  strong 
assumptions  about  human  rationality  and  problem-solving  abilities  enable  modelers  to  solve  the 
calculus  used  in  economic  models.  By  contrast,  ABMs  can  represent  humans  in  a  more  natural  way, 
known  as  "bounded  rationality"  (Simon,  1972).  In  these  models  humans  can  behave  like  humans 
and  make  mistakes,  come  to  incorrect  conclusions,  have  prejudices,  and  act  based  upon 
incomplete  information. 

Just  as  important,  unusual  events  and  rare  individuals  -  so-called  outliers  or  "black  swans"  (Taleb, 
2010)  -  can  have  a  profound  impact  on  the  system.  Many  other  analytic  tools  used  to  study  human 


Computational  sociocultural  models  used  for  forecasting  |  289 


systems  ignore  or  define  away  these  outliers.  This  can  make  the  impact  of  rare  events  such  as 
terrorism  or  the  influence  of  unusually  charismatic  individuals,  as  part  of  a  "normal"  population, 
very  difficult  to  understand,  and  highlights  the  shortcomings  of  representing  heterogeneous 
systems  in  analytic  tools  other  than  ABMs. 

6.2.  Using  Agent-Based  Models 

Like  all  simulation  and  analytic  tools,  ABMs  can  be  used  correctly  or  incorrectly.  Two  important 
frameworks  have  been  built  up  around  the  use  of  ABMs:  "docking"  (Axtell,  Axelrod,  Epstein,  & 
Cohen,  1996);  and  "Levels  of  Empirical  Relevance"  (LER)  (Axtell,  2005).  Docking  describes  how  well 
an  ABM  relates  to  a  reference.  As  shown  in  Figure  14,  there  are  three  levels  of  docking:  Identity 
(the  ABM  produces  results  identical  to  the  reference),  Distributional  (the  ABM  produces  results 
statistically  indistinguishable  from  the  reference),  and  Relational  (the  ABM  behaves  in  a  manner 
consistent  with  the  reference  but  the  results  are  statistically  distinguishable).  LER  has  four  levels. 
At  Level  0  the  agents  within  the  ABM  behave  in  a  plausible  manner  for  a  given  domain.  At  Level  1 
the  ABM  produces  overall  reasonable  dynamics;  e.g.,  firms  of  different  sizes  exist  in  an  artificial 
economy.  At  Level  2  the  ABM  produces  the  correct  overall  structure;  e.g.,  the  distribution  of  firm 
sizes  in  the  artificial  economy  corresponds  to  the  distribution  in  the  real  economy  (Axtell,  2001). 
Finally,  at  Level  3  the  ABM  produces  correct  results  at  the  individual  level;  e.g.,  the  artificial  firms 
represent  real  firms  and  both  firms  are  the  same  size. 

As  the  figure  shows,  combining  docking  and  LER  creates  a  framework  that  guides  how  carefully 
researchers  must  evaluate  the  ABM  and  how  much  time  and  effort  it  will  take  to  create  the  ABM. 
At  present  the  data  requirements  for  Level  3  LER  (correct  results  at  the  individual  level)  are  so 
onerous  that  it  is  very  unlikely  that  an  ABM  can  be  created  to  achieve  this  level.  Advances  in  big 
data,  data  sharing,  and  social  computing  will  mitigate  this  limitation;  however,  for  many  topics  and 
regions  of  national  security  interest  the  constraint  will  persist  for  some  time.  Researchers  must  also 
take  the  scale  of  the  system  and  of  the  ABM  into  account.  These  sorts  of  systems  can  change  their 
behavior  as  the  number  of  entities  increases  (for  example,  a  single  water  molecule  cannot  produce 
"wetness";  that  property  only  belongs  to  large  groups  of  water  molecules).  The  same  is  true  for 
ABMs:  an  ABM  cannot  be  considered  satisfactory  until  it  functions  correctly  at  the  same  (or  as 
similar  as  possible)  scale  as  the  system  being  studied. 
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A  Basic  System  for  Evaluating  Agent-Based  Models 
and  Thinking  About  Their  Appropriate  Use 
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Figure  14.  Basic  system  for  evaluating  ABMs  and  their  appropriate  use. 


6.3.  Using  Agent-Based  Models  for  Forecasting 

ABMs  are  typically  used  for  explanatory  models  rather  than  forecasting  per  se.  ABMs  demonstrate 
what  conditions  suffice  to  generate  a  particular  outcome,  but  do  not  predict  when  the  outcome 
will  happen.  For  example,  an  analyst  could  use  an  ABM  to  explore  the  conditions  (armaments, 
visibility  conditions,  levels  of  aggression,  communications  gear,  etc.)  under  which  a  U.S.  Marine 
Corps  (USMC)  fire  team  could  defeat  an  enemy  tank  platoon  (which  has  its  own  set  of  armaments, 
aggression,  and  so  on).  Running  this  ABM  many  times  would  generate  a  large  number  of  results 
that  could  then  be  binned  by  numbers  of  USMC  casualties  or  by  "wins"  and  "losses."  The  analyst 
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would  then  look  for  commonalities  among  runs  that  fell  into  the  bins  of  interest.  In  this  example, 
the  ABM  helped  the  analyst  to  understand  a  situation  of  interest,  not  to  identify  a  situation  of 
interest  or  forecast  that  situation's  likelihood  of  occurring.  The  context  for  the  use  of  an  ABM 
becomes,  "If  we  find  ourselves  in  situation  X,  then  we  can  expect  these  sorts  of  outcomes."  When 
thinking  about  outcomes  and  rates  of  occurrences  or  likelihoods,  it  should  be  noted  that  the 
outcomes  of  ABMs  are  often  not  shaped  like  a  bell  curve.  This  means  that  traditional  notions  of 
likelihood  (averages)  may  not  be  meaningful  and  should  be  carefully  considered  when  used  for  risk 
and  course  of  action  (COA)  analyses. 

The  fire  team  offers  a  fairly  simple  example,  as  it  involves  relatively  few  agents  and  they  interact  in 
very  specific  ways.  Constructing  an  ABM  becomes  significantly  more  difficult,  and  the  data 
requirements  become  more  onerous,  in  more  "open"  sociocultural  realms.  However,  the  time  and 
data  requirements  can  be  tempered  by  increasing  the  level  of  abstraction.  For  example,  the  1st 
Marine  Expeditionary  Force  (1MEF)  used  an  ABM  to  examine  how  a  population's  level  of 
contentment  would  change  as  "good"  and  "bad"  events  happened  in  the  area  of  operations 
(Koehler,  Barry,  Widdowson,  &  Forsyth,  2004).  Changes  in  the  population's  contentment  were  a 
function  of  the  group  that  the  population  blamed  for  the  events  and  the  population's  attitude 
toward  that  group,  as  well  as  the  contentment  of  the  population's  social  network.  This  ABM  had 
minimal  data  requirements,  as  1MEF  was  interested  in  the  dynamics  of  a  relatively  generic 
population;  therefore,  only  the  internal  mechanisms  of  attitude  change  had  to  be  well  specified 
and  neither  geospatial  data  nor  specific  demographic  data  were  needed.  This  also  meant  that 
1MEF  could  accomplish  the  work  in  weeks,  producing  a  model  that  was  LER  Level  0. 

The  Joint  Improvised  Explosive  Device  (IED)  Defeat  Organization  (JIEDDO)  has  also  used  ABMs  for 
understanding  sociocultural  dynamics;  however,  JIEDDO  wanted  to  understand  the  impact  of  these 
dynamics  on  IED  use  (Turnley,  Henscheid,  Koehler,  Mulutzie,  &  Tivnan,  2012).  This  meant  the  ABM 
had  to  simulate  a  specific  time  and  place  with  a  specific  population  and  specific  geometry,  as  well 
as  specific  coalition  and  insurgent  concepts  of  operations.  The  data  requirements  and  time  needed 
to  create  the  model  were  both  quite  high;  the  effort  produced  a  model  that  was  LER  0-LER  2, 
depending  on  the  model  component.  However,  JIEDDO  could  use  the  model  to  understand  the 
complex  interrelationships  among  coalition  activities,  insurgent  activities,  traditional  power 
networks,  and  geospatial  factors,  among  others. 


7.  Hybrid  Model  Forecasts 

A  model  that  combines,  for  example,  system  dynamics  and  ABM  approaches  can  be  defined  as  a 
hybrid  model.  According  to  Prasad  &  Park  (2000),  this  "approach  represents  a  middle  ground 
between  the  analog  modeling  based  on  differential  equations  in  system  dynamics  and  the  discrete 
rule-based  modeling  methods  common  in  the  agent  based  simulation  community."  System 
dynamics  assumes  homogeneity  of  individuals,  while  ABMs  relax  aggregation  assumptions  and  can 
capture  individual  heterogeneity.  However,  the  complexity  of  ABMs  increases  computational 
requirements;  further,  ABMs  can  be  difficult  to  construct  and  to  validate  (Rahmandad  &  Sterman, 
2008).  ABMs  with  embedded  system  dynamics  models  can  overcome  some  of  these  difficulties  and 
may  provide  new  insights  into  the  systems  modeled  (Teose  et  al.,  2011). 
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7.1.  Hybrid  Insurgency  Dynamics  Model 

This  section  uses  the  hybrid  Insurgency  Dynamics  model  to  illustrate  how  such  a  model  can  support 
forecasts.  The  original  model  was  developed  at  the  Massachusetts  Institute  of  Technology  (Choucri, 
Goldsmith,  Madnick,  Mistree,  Morrison,  &  Siegel,  2007).  The  AnyLogic  Company  (2013)  added 
agents  and  agent  behavior  to  this  model  and  distributed  it  as  a  sample  model  with  their  software 
package.  The  agent-based  portion  of  the  model  captures  simple  agent  interaction  for  recruitment, 
and  is  integrated  with  the  system  dynamics  portion,  which  captures  high-level  relationships.  We 
adapted  this  model  further  using  features  from  the  Counterinsurgency  (COIN)  model  (Turnley  et 
al.,  2012)  that  is  based  on  Epstein's  (2002)  work  on  models  of  civil  violence.  Updates  enabled  the 
agents  to  belong  to  social  groups  and  interact  with  others  both  within  and  outside  their  respective 
groups.  These  interactions  affect  the  agent's  beliefs  and  behavior.  Next  we  augmented  agents  with 
emotion,  specifically  characterizing  the  relationship  between  anger  and  contempt  and  their 
influence  on  an  agent's  tendency  toward  non-violent  and  violent  direct  action  (Jost,  Chaikalis- 
Petritsis,  Abrams,  Sidanius,  van  der  Torn,  &  Bratt,  2011;  Tausch,  Beckerm,  Spears,  Christ,  Saab, 
Singh,  &  Siddiqui.,  2011). 

We  use  groups  consisting  of  agents  with  similar  social  characteristics  (e.g.,  ethnicity,  political 
beliefs,  and  religious  beliefs).  Initial  attributes  are  based  on  a  group's  relationship  to  other  groups 
(perceived  legitimacy,  perceived  hardship,  grievances).  Agents  interact  at  variable  rates,  but 
interaction  among  group  members  occurs  at  higher  rates  within  groups  than  across  groups.  When 
one  agent  interacts  with  another,  the  model  adjusts  each  agent's  activation  value  (a  personal 
threshold  for  taking  violent/non-violent  direct  action)  based  on  the  other's  activation  values. 

Each  agent  can  be  in  one  of  four  states:  government  supporter ,  dissident  or  protestor,  insurgent  or 
violent  protestor,  and  removed  insurgent,  as  shown  on  the  right  of  Figure  15.  Whether  a 
government  supporter  becomes  a  dissident  is  based  on  the  agent's  activation  values  (i.e., 
thresholds  set).  The  model  calculates  initial  activation  values  for  each  agent  at  run  time  based  on 
agent  emotions,  perceived  legitimacy  of  other  agents  or  groups,  perceived  hardship,  and 
grievances.  An  agent  in  a  dissident  state  may  go  back  to  being  a  non-activist  or  government 
supporter  after  a  certain  amount  of  time  or  when  the  agent's  activation  value  falls  below  the 
threshold.  However,  an  agent  that  becomes  an  insurgent  cannot  go  back  to  the  dissident  state;  the 
agent  remains  in  the  insurgent  state  until  transitioned  to  the  removed  insurgent  state  at  which 
point  the  agent  is  removed  from  the  model. 

Emotion  values,  from  non-traditional  data  source  Twitter,  for  anger,  contempt,  negative  emotion, 
and  positive  emotion  are  integrated  into  the  model  daily— the  model  has  a  daily  time  step.  For 
example,  values  for  agents  within  a  group  (i.e.,  not  individual)  are  adjusted  based  on  group  values 
and  affect  their  activation  state.  The  model  normalizes  emotion  strength  for  the  number  of  tweets 
(ratio  of  number  of  tweets  reflecting  a  particular  emotion  on  a  given  day  to  the  total  number  of 
tweets  reflecting  all  the  emotions  being  considered  on  that  same  day).  For  instance,  the  model 
adds  the  value  for  anger  to  the  agent's  activation  value  as  a  condition  for  transitioning  from  a 
government  supporter  to  a  dissident  state,  and  adds  the  contempt  value  as  a  condition  for 
transitioning  from  dissident  to  insurgent.  Other  attributes  based  on  traditional  data  sources  include 
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attitude  toward  risk,  risk  aversion,  and  threshold  of  willingness  to  become  a  peaceful  and/or 
violent  activist. 

The  system  dynamics  component  contains  the  global  parameters  for  the  model,  as  shown  on  the 
left  side  of  Figure  15.  Integration  points  between  the  system  dynamics  and  agent-based 
components  occur  through  parameters:  propensity  to  be  recruited,  number  of  insurgents,  number 
of  dissidents,  and  number  of  government  supporters.  The  individual  agent  state  charts  shown  on 
the  right  of  Figure  15  incorporate  each  of  these  parameters.  In  summary,  the  system  dynamics 
parameters  are  adjusted  by  tracking  the  changing  population  of  agents. 

7.2.  Use  Case 

Following  the  2007  Kenyan  presidential  election,  violence  broke  out  when  supporters  of  the 
defeated  candidate  perceived  the  outcome  as  unfair.  The  conflicts  exacerbated  divisiveness  among 
tribes;  thousands  of  persons  were  killed  and  many  more  displaced  from  their  homes.  In  2010, 
Kenya  adopted  a  new  national  constitution  and  in  2012,  during  the  run-up  to  the  next  election,  two 
candidates,  Uhuru  Kenyatta  and  Raila  Odinga,  emerged  as  the  frontrunners  for  elections  in  2013. 
The  government  and  people  of  Kenya,  neighboring  nations,  and  the  United  States  hoped  to  see 
non-violent  execution  of  the  historical  election.  We  mapped  short-term  forecasts  of  dynamic 
emotion  indicators  to  parameters  in  a  theory-based  hybrid  model,  as  described  below,  to  monitor 
instability  that  might  affect  the  2013  Kenyan  elections. 

7.3.  Dynamic  Emotion  Data 

This  study  used  Twitter  data  to  dynamically  update  model  parameters  in  near-real  time,  support 
indicator  integration  (anger,  contempt,  positive  emotion,  and  negative  emotion),  and  identified 
potential  threshold  values  to  signal  when  an  analyst  should  investigate  further.  To  this  end,  we 
collected  Twitter  data  sets  on  Kenyan  election-related  keywords,  hash  tags,  and  usernames 
between  December  2012  and  April  2013.  The  two  political  coalition-level  Twitter  data  sets  focused 
on  Odinga's  coalition,  the  Coalition  for  Reforms  and  Democracy  (CORD)  (for  example,  #cord, 
#kalonzo,  #odinga,  Kalonzo,  ODM-Kenya,  Wiper)  and  Kenyatta's  coalition,  "Jubilee"  (for  example, 
#kenyatta,  #narc,  @williamsruto,  Jubilee  alliance,  Kenyatta,  NPK,  Ruto,  Uhuruto). 
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Figure  15.  System  dynamics  and  agent-based  components  integrated  through  parameters. 
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The  emotion  analysis  indicators  from  the  UWC  approach  (Elson,  Yeung,  Roshan,  Bohandy,  &  Nader, 
2012;  Servi  &  Elson,  2012)  enabled  the  use  of  dynamic,  near-real  time  data  in  the  model.  The  UWC 
algorithm  calculates  the  ratio  of  words  for  one  particular  category  to  the  total  word  count  for  a 
given  set  of  tweets.  The  following  UWC  emotion  indicators  from  Twitter  were  integrated  into  the 
simulation  model  based  on  available  social  science  theory  (Figure  16):  anger,  contempt,  positive 
emotion,  and  negative  emotion.  The  forecasted  values  could  be  constant  (averages)  or  change  with 
time  (trends)  and  are  described  next. 

7.4.  Short-Term  Forecast  of  Data 

After  mapping  the  near-real  time  data  series  to  the  hybrid  model,  we  needed  a  short-term  forecast 
of  the  indicators  to  evaluate  model  output.  Box  and  Jenkins  (1970)  provide  techniques  for  building 
statistical  models  with  the  minimum  number  of  parameters  as  described  below.  Their  approach 
involves  taking  observations  at  discrete,  equally  spaced  intervals  of  time  and  making  forecasts  for 
future  time  intervals.  The  approach  also  provides  ways  to  calculate  probability  limits  on  either  side 
of  the  forecasted  values,  for  example,  at  the  ±95%  confidence  range. 

Autoregressive  models  (ARs)  express  the  current  value  of  a  process  as  a  finite,  linear  combination 
of  previous  values  of  the  process  and  a  random  value.  A  first-order  AR  model  uses  one  point  and 
predicts  values  using  a  single  coefficient.  A  second-order  AR  model  of  the  same  data  would  have 
two  coefficients  and  use  two  data  points.  Moving  Average  (MA)  models  take  a  linear  combination 
of  the  past  residuals  to  form  a  prediction.  The  difference  between  the  prediction  and  the  actual 
value  is  called  a  residual,  and  the  model  fit  is  measured  as  the  variance  in  the  residuals.  In  a  well¬ 
fitting  model  the  residuals  should  be  distributed  as  white  noise,  i.e.,  normally  distributed  with  zero 
mean  and  a  variance. 

The  AR  and  MA  models  can  be  combined  and  applied  to  actual  values  in  the  time  series  data  to 
generate  an  average  forecast  or  applied  to  differences  in  the  data  values  to  generate  a  trend 
forecast— the  latter  is  called  an  Autoregressive  Integrated  Moving  Average  (ARIMA).  The  order  of 
the  models  is  described  by  three  values:  the  number  of  parameters  in  the  AR,  the  number  of 
differences,  and  the  order  of  the  MA.  We  used  an  ARIMA  model  with  two  parameters,  one 
difference,  and  one  order  of  the  moving  average.  The  shaded  region  on  the  right  of  the  bottom 
graphs  in  Figure  16  show  the  forecast  for  the  four  emotions  for  the  ±95%  confidence  range. 

7.5.  Sensitivity  Analysis 

We  performed  sensitivity  analysis  by  holding  one  series  of  emotion  values  constant  over  time  while 
all  others  were  allowed  to  vary  as  measured  for  the  Kenyan  elections.  This  process  was  repeated 
for  a  range  of  values,  running  each  case  once.  The  model  parameters  and  group  attributes  run  in 
the  sensitivity  analysis  included  anger,  contempt,  probability  of  arrest,  appeasement,  removal 
effectiveness,  risk  aversion,  positive  emotion,  negative  emotion,  contact  rate,  average  time  as 
dissident,  and  desired  time  to  remove  insurgents.  The  number  of  dissidents  was  recorded  as  a 
function  of  time. 

During  the  sensitivity  analysis,  we  observed  thresholds  of  emotion  in  which  the  number  of 
dissidents  increased  or  decreased.  For  example,  increased  anger  (values  tested  0.001-0.009)  in  the 
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CORD  dataset  did  not  correspond  to  an  increase  in  the  number  of  dissidents  in  the  model; 
however,  in  the  Jubilee  dataset  the  number  of  dissidents  rose  as  anger  rose.  Specifically,  after  April 
10,  2013  the  number  of  CORD  dissidents  decreased  to  zero  for  all  values  of  anger,  but  for  a 
constant  value  of  anger  greater  than  0.003  the  number  of  Jubilee  dissidents  increased,  as  shown  in 
the  top  of  Figure  16.  This  revealed  that  the  Jubilee  data  set  is  more  sensitive  to  anger— it  did  not 
decrease  to  zero.  In  summary,  using  the  highest  anger  value  tested  (0.009)  in  the  model  was  not 
enough  to  create  CORD  dissidents  but  values  greater  than  0.003  anger  created  dissidents  in  the 
Jubilee  dataset.  The  reason  this  happens  is  due  both  to  the  emotions  in  the  datasets  (CORD  vs. 
Jubilee)  and  model  structure.  Positive  and  negative  emotion  contributes  to  an  agent's  propensity 
to  be  recruited,  which  impacts  an  agent's  threshold  for  becoming  a  dissident.  Therefore,  in  the 
CORD  dataset,  the  values  for  positive  and  negative  emotion  offset  the  anger  response.  Similar 
thresholds  can  be  extracted  from  the  positive  and  negative  emotion  results;  however,  the 
contempt  words  were  not  evident  in  these  data  sets,  as  illustrated  in  the  bottom  of  Figure  16. 

7.6.  Conclusions 

Fortunately,  the  2013  Kenyan  elections  proved  largely  peaceful.  Actual  US  courses  of  action  (COAs) 
included  Department  of  State  messages  and  President  Obama's  video  message  in  February  2013 
wishing  Kenyans  a  peaceful  election.  The  two  data  sets  showed  little  correlation  between  these 
messages  and  the  UWC  emotions,  because  the  data  sets  focused  on  immediate  party  politics  and 
leaders. 

This  study  used  Twitter  data  to  dynamically  update  model  parameters  in  near-real  time  to  support 
indicator  integration  (anger,  contempt,  positive  emotion,  and  negative  emotion)  to  determine 
potential  threshold  values  to  signal  that  an  analyst  should  investigate  further.  Using  a  theory-based 
model  with  short-term  forecasts  of  the  emotion  indicators  to  determine  the  degree  of  emotion 
necessary  to  trigger  an  increasing  number  of  dissidents  is  our  contribution  and  requires  further 
work.  Other  future  work  should  include  calibrating  the  potential  COAs  for  the  amount  of  change 
necessary  to  counter  the  trend  in  the  number  of  dissidents.  The  largest  research  gaps  include 
instantiating  social  science  theory  in  simulation  models  including  identifying  other  theories  to  test; 
obtaining  robust,  meaningful,  dynamic  datasets  to  update  theory-based  models;  and  determining 
how  best  to  use  the  data  in  the  modeling  process. 
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Figure  16.  UWC  indicators  over  time  with  a  5-day  short-term  forecast  (bottom)  and  sensitivity 
analysis  to  determine  thresholds  for  emotion  indicators  (top). 
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8.  Evolving  Self-Organizing  Maps  (ESOM) 

Self-Organizing  Maps  (SOM)  is  a  soft-computational  technique  for  visualizing  high-dimensional  data 
(Kohonen,  2001).  According  to  McGaugh  (2012),  SOM  addresses  flexibility  issues  by  processing 
data  iteratively  to  enable  the  best  mathematical  representation  of  data  while  preserving  the 
underlying  structure  of  the  data.  Complex  systems  can  be  analyzed  and  modeled  using  SOM  since  it 
uses  unsupervised  learning  algorithms  thereby  allowing  clustering  of  the  data  without  requiring  the 
modeler  to  know  the  class  membership  function  of  the  input  data  (Simula,  Vesanto,  Alhoniemi,  & 
Hollmen,  1999).  Deng  and  Kasabov  (2000)  proposed  an  algorithm  of  Evolving  Self  Organizing  Map 
(ESOM)  in  which  one  passes  learning  on  in  the  dynamic  version  of  the  Kohonen  SOM.  SOM  evolves 
over  time,  adapting  itself  to  changing  data  streams  by  providing  a  method  for  good  clustering,  data 
analysis,  visualization,  and  prediction.  This  algorithm  can  be  extended  for  supervised  learning  by 
augmenting  input  vectors  with  target  output  variables  during  the  training  session.  The  output  value 
of  the  best  matching  unit  (BMU)  of  the  trained  model  is  then  substituted  for  the  predicted  value 
for  the  input  vector  without  augmentation  for  which  prediction  has  to  be  made  (Deng  &  Kasabov, 
2000). 

Among  the  SOM  algorithms  investigated,  ESOM  yielded  the  most  promising  results.  ESOM  starts 
with  the  model  set  to  zero  output  nodes.  As  an  n-dimensional  input  vector  is  presented  to  the 
model,  the  algorithm  determines  the  Euclidian  distance  from  all  the  existing  output  nodes.  If  no 
output  node  exists  (the  condition  when  the  first  input  is  received  for  training),  the  algorithm 
creates  a  new  output  node  in  the  image  of  the  current  input  nodes.  Thus  the  output  nodes  have 
the  same  dimensions  as  the  input  vector.  If  the  smallest  distance  between  the  current  input  vector 
and  output  nodes  is  greater  than  threshold  value  £  the  algorithm  creates  a  new  output  node  in  the 
image  of  the  current  input  vector.  If  the  smallest  Euclidean  distance  between  the  current  input 
vector  and  output  nodes  less  or  equal  to  £,  the  output  node  that  has  the  minimum  distance  from 
the  current  input  vector  is  called  the  Best  Matching  Unit  (BMU).  The  algorithm  updates  the  location 
of  the  BMU  resulting  in  it  becoming  closer  to  the  current  input  vector.  Additionally,  it  updates  the 
two  closest  neighbors  of  the  BMU. 

After  presenting  all  input  vectors  and  the  training  phase  terminates,  the  ESOM  model  can  be  used 
for  testing  and  prediction.  ESOM  can  be  applied  to  supervised  learning  tasks  such  as  classification 
and  prediction  as  described  by  Deng  and  Kasabov  (2000).  Results  from  the  study  of  Afghanistan 
(Ahram  &  Karwowski,  2012)  showed  that  ESOM  can  help  to  forecast  adverse  events  by  drawing  on 
knowledge  of  selected  socioeconomic  factors  as  well  as  information  about  the  recent  adverse 
events  that  occurred  in  a  given  region. 

8.1.  Artificial  Neural  Networks  (ANNs) 

Artificial  neural  networks  (ANNs)  are  mathematical  models  of  the  human  brain  that  can  identify 
complex  nonlinear  and  dynamic  relationships  between  input  and  output  datasets.  According  to 
Choubey,  Sehgal,  and  Tandon  (2006),  the  Multilayer  Perceptron  (MLP)  is  the  neural  network  model 
type  most  commonly  described  in  the  literature.  An  MLP  is  a  type  of  supervised  network  as  it  needs 
a  desired  output  in  order  to  learn.  This  type  of  network  uses  historical  input  and  output  to  build  a 
model  that  correctly  maps  the  future  real  data  to  the  appropriate  output. 
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According  to  Assi,  Shamisi,  and  Hejase  (2012),  a  graphical  representation  of  an  MLP  has  three  types 
of  layers:  input  layer,  hidden  layer,  and  output  layer,  as  shown  in  Figure  17.  Neurons  in  the  input 
layer  distribute  the  input  signals  to  neurons  in  the  hidden  layer.  Each  neuron  in  the  hidden  layer 
sums  up  its  input  signals  after  assigning  them  the  weights  of  the  respective  connections  from  the 
input  layer.  This  weighted  average  is  then  applied  to  a  simple  threshold  function,  sigmoidal 
function,  hyperbolic  tangent  function,  or  radial  basis  function  to  obtain  the  output.  The  back 
propagation  algorithm  is  the  most  commonly  adopted  MLP  training  algorithm.  It  changes  the 
weight  of  the  connection  between  each  pair  of  neurons  (Assi  et  al.,  2012)  based  on  the  desired 
learning  rate  and  the  difference  between  the  target  output  and  the  actual  output  as  shown  in 
Figure  18. 


Input  layer  Hidden  layer 


Figure  17.  Architecture  of  a  multilayered  neural  network  (adapted  with  permission  from  Dayhoff  & 
DeLeo,  2001). 


Figure  18.  Typical  artificial  neuron  (adapted  with  permission  from  Dayhoff  &  DeLeo,  2001). 
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8.2.  Afghanistan  Study 

The  Ahram  and  Karwowski  (2012)  study  of  Afghanistan  used  these  soft  modeling  methods  to 
generate  forecasts  that  would  guide  allocation  of  development  funds  for  stabilizing  regions  with 
high  levels  of  adverse  events  and  to  better  understand  the  complex  socioeconomic  interactions 
caused  by  adverse  events.  The  Department  of  Defense's  Human  Social  Culture  Behavior  (HSCB) 
Modeling  program  provided  two  different  types  of  datasets  collected  between  2002  and  2010.  The 
study  used  a  set  of  spatial  data  representing  social  and  economic  factors  from  Worldwide 
Incidents  Tracking  System  Geography  (WITS  GEO),  the  U.S.  Agency  for  International 
Development  (USAID),  and  AISCS  (African  Immigrants  Social  &  Cultural  Services).  The  adverse  event 
dataset  included  information  regarding  the  date  of  an  event;  incident  type;  number  of  people 
killed,  wounded,  and  kidnapped;  province,  city,  and  district;  description  of  the  event,  and  simple 
event  summary.  The  infrastructure  aid  dataset  included  the  urban  and  rural  population  density, 
province,  city,  district,  country,  project  types,  allocated  budget  information  for  different  sectors, 
amount  of  aids  in  each  sector,  types  of  construction,  and  usage  of  airfields.  The  model  outputs 
included  adverse  events  in  terms  of  the  number  of  people  killed,  wounded,  hijacked,  and  total 
number  of  adverse  events. 

The  study  divided  Afghanistan  into  seven  regions  for  analysis  purposes.  For  each  model,  the  data 
from  2002-2009  were  grouped  and  used  for  training,  while  the  2010  data  were  used  for  testing. 
The  researchers  investigated  the  performance  of  each  of  the  96  models  using  the  Mean  Absolute 
Error  (MAE)  and  the  prediction  accuracy  within  ±1  error  range  (difference  between  actual  and 
predicted  value).  The  sensitivity  analysis  showed  the  importance  of  the  relationship  between  the 
varied  economic  development  projects  on  geographical  regions,  population  density,  and 
occurrence  of  adverse  events  in  Afghanistan. 


9.  Forecasts  of  Social  Networks  Generated  from  Text  Analysis 

For  research  questions  that  involve  dynamics,  such  as  explaining  how  networks  emerge,  how 
information  diffuses  through  networks,  how  a  community's  beliefs  change  as  a  function  of  network 
changes,  or  how  networks  evolve,  computer  simulation  is  generally  used.  Most  of  these  social 
network  simulation  models  are  aimed  at  understanding  or  predicting  the  spread  of  information, 
beliefs,  goods,  or  diseases  through  networks.  As  such  most  of  the  models  take  into  account  some 
measure  of  network  exposure  -  the  likelihood  that  an  actor  will  be  exposed  to  something  because 
of  the  number  of  others  that  they  are  linked  to  who  already  have  been  exposed.  Most  such  models 
take  into  account  social  influence,  i.e.,  the  extent  to  which  an  actor  is  likely  to  behave  in  a  certain 
way  or  hold  a  certain  belief,  because  others  to  which  that  actor  already  is  linked  behave  in  that  way 
or  hold  that  belief.  Consequently,  most  such  models  also  have  a  way  of  predicting  cascades,  i.e., 
the  trace  of  change  in  who  knows  or  has  what  as  a  function  of  an  initial  exchange. 

Agent-based  simulation  is  considered  a  natural  for  simulating  networks.  In  an  agent-based  model, 
social  behavior  emerges  from  the  actions  of  and  interactions  among  the  individual  simulated 
actors,  i.e.,  the  agents.  In  most  agent-based  models,  all  agents  are  of  one  type,  e.g.,  all  people  or  all 
organizations,  and  the  activity  often  does  not  take  into  account  actual  social  network  constraints, 
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use  of  communication  technology  or  the  role  of  media  including  social  media,  or  geographic  issues. 
Social  networks  are  often  inferred  by  which  other  agents  an  agent  comes  into  contact  with  based 
on  their  position  in  a  grid.  Such  grid  based  networks  are  not  similar  to  natural  human  social 
networks  in  that  they  tend  not  to  have  appropriate  distributions  of  strong  and  weak  ties,  degree 
centrality  and  reciprocity. 

In  contrast,  to  traditional  agent-based  models,  agent-based  dynamic-network  models  use  or 
generate  networks  that  are  more  socially  realistic.  In  an  agent-based  dynamic-network  model,  the 
agents  are  not  located  in  grids  but  in  networks.  Moreover,  these  networks  may  evolve  over  time; 
i.e.,  who  is  connected  to  whom  and  the  strength  of  such  connections  may  change.  An  example  of 
such  an  agent-based  dynamic-network  model  is  CONSTRUCT. 

CONSTRUCT  is  an  agent-based  dynamic-network  simulation  framework  that  can  aid  assessments  of 
changes  in  the  distribution  of  knowledge  and  sentiment  at  the  social  level  (Carley,  Martin,  & 
Hirshman,  2009).  It  is  an  extension  beyond  the  standard  ABMs  previously  described.  Based  on  the 
constructural  theory  of  sociocultural  structuration  (Carley,  1990),  the  CONSTRUCT  model  supports 
reasoning  about  the  social  network  and  learning  dynamics  that  underlie  the  diffusion  of 
information  and  dispersion  of  beliefs  as  the  networks  evolve.  CONSTRUCT  is  the  first  agent-based 
model  in  which  the  agents  are  distributed  in  high-dimensional  social  and  homophily  networks,  such 
that  the  networks  need  not  be  static  but  instead  co-evolve  with  the  knowledge  networks.  The 
model  has  been  validated  in  numerous  settings  including  predicting  the  impact  of  inter-marriage, 
leadership,  and  change  in  terror  groups  (Schreiber  &  Carley,  2004). 

9.1.  Basic  Constructural  Model 

CONSTRUCT  takes  as  input  the  initial  observed  social,  knowledge  and  belief  networks,  as  well  as 
communication  media.  Input  can  also  include  other  homophily  characteristics  in  terms  of  socio¬ 
demographic  characteristics  of  the  population.  The  social  network  defines  who  interacts  with 
whom,  the  knowledge  network  (who  knows  what),  and  the  belief  network  (who  believes  what). 
Agents  have  a  "transactive"  memory  of  other  agents;  thus,  each  agent  has  a  perception  of  what 
others  know  and  believe  and  with  whom  they  are  likely  to  interact.  Agents  can  communicate  those 
perceptions  -  and  those  perceptions  can  be  wrong. 

CONSTRUCT  can  be  instantiated  using  data  on  either  real  or  hypothetical  groups,  and  therefore 
using  real  or  artificial  data.  To  gather  real  data,  as  was  done  in  Carnegie  Mellon  University's 
"SUDAN  Game"  (Landwehr,  Sparagen,  Ranganathan,  Carley,  &  Zyda,  2012),  researchers  can  mine 
the  text  of  news  or  social  media  to  extract  the  set  of  actors,  topics  (often  characterized  as 
knowledge),  beliefs,  and  the  relations  among  them.  Those  data  function  as  time  "0"  data;  i.e.,  the 
starting  case.  CONSTRUCT  then  simulates  the  system  and  "updates"  the  data  to  show  how  they 
may  look  at  time  "A/";  i.e.,  at  some  later  date.  To  facilitate  instantiation  of  the  CONSTRUCT  model, 
analysis  of  the  results,  and  validation  of  the  results  against  real  data,  CONSTRUCT  interoperates 
with  other  tools,  such  as  AutoMap,  a  text  mining  tool  that  extracts  networks  in  DyNetML,  and  ORA, 
a  network  analysis  tool  that  reads  and  writes  DyNetML.  Finally,  ORA  can  process  and  analyze  the 
CONSTRUCT  results  and  compare  them  with  other  data  in  the  same  format. 
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While  the  networks  view  the  homophily  characteristics  as  unchanging,  at  least  in  the  short  run, 
they  have  the  potential  to  evolve.  This  evolution  is  a  function  of  the  communication-learning  cycle 
that  all  agents  undergo  more  or  less  simultaneously.  This  means  that  two  agents  will  have  a 
number  of  characteristics  in  common  -  e.g.,  age  and  gender  -  that  will  persist  over  time,  and  may 
also  share  certain  information  or  beliefs.  As  these  agents  interact  they  will  learn  new  information, 
learn  what  others  believe,  and  therefore  change  what  they  know  and  possibly  believe.  Thus  the 
similarity  of  these  agents  in  knowledge  and  beliefs  may  change. 

Whether  or  not  two  agents  interact  is  a  function  of  their  position  in  the  social  network  and  the 
available  communication  media.  Agents  are  more  likely  to  interact  with  those  to  whom  they  have 
strong  social  ties.  However,  they  occasionally  interact  with  those  to  whom  they  are  weakly  tied. 
Depending  on  the  communication  media  available,  these  interactions  occur  simultaneously  or 
asynchronously.  For  example,  agents  with  access  to  email  can  send  a  message  to  multiple  other 
agents  simultaneously,  but  the  other  receivers  may  receive  that  message  at  different  times.  By 
contrast,  in  a  face-to-face  interaction  both  agents  must  be  co-present  and  two  agents  participate, 
unless  it  is  a  group  meeting. 

The  social  network  used  for  interaction  is  the  underlying  "true"  social  network,  defined  as  a 
distribution  of  interaction  probabilities.  These  probabilities  depend  on  any  or  all  of  the  following: 
how  much  information  the  two  individuals  share;  how  similar  the  two  individuals  are  in  their 
beliefs;  how  much  information  the  one  individual  thinks  the  other  has,  i.e.,  called  "ego";  similarity 
in  socio-demographics;  and  organizational  or  physical  constraints  on  the  need  to  interact  or  that 
prevent  interaction.  In  general,  each  of  these  factors  is  defined  relative  to  the  individual's  similarity 
or  difference  with  all  other  actors  including  his  or  herself.  These  factors  are  used  to  construct  the 
relative  similarity  between  two  individuals,  taking  into  account  the  observed  network,  shared 
knowledge  and  beliefs,  and  socio-demographic  similarities.  This  relative  similarity  is  treated  as  an 
interaction  probability. 

When  agents  interact,  they  communicate.  The  complexity,  and  to  some  extent  the  content,  of 
those  communications  vary  based  on  the  communication  media;  for  example,  Twitter  messages 
are  less  complex  than  books  or  phone  calls.  As  agents  interact  and  communicate  they  learn  the 
new  information  or  beliefs,  with  possible  error  and  with  forgetting.  At  each  time  step  the  model 
updates  the  probability  of  interaction  between  each  pair  of  agents  based  on  the  evolving  set  of 
information  and  beliefs.  Thus,  in  the  long  run,  and  with  sufficiently  large  enough  bodies  of 
knowledge  and  beliefs,  whether  or  not  two  agents  interact  will  become  more  a  function  of  the 
learned  information  than  of  the  initial  observed  network  and  demographic  properties.  However,  in 
the  short  run,  the  existing  structure  and  the  socio-demographics  dominate. 

Learning  has  a  key  impact  on  the  agents:  it  changes  which  other  agents  they  are  likely  to  interact 
with  in  the  future  and  what  they  can  communicate.  In  this  sense,  the  social,  knowledge,  and  belief 
networks  in  CONSTRUCT  co-evolve.  As  agents  communicate,  learn,  and  change  their  positions  in 
their  social  networks,  groups  can  emerge  and  disappear,  culture  can  shift,  and  individual  agents 
can  increase  or  decrease  in  prominence. 
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9.2.  Illustrative  Results 

Consider  the  following  three  examples:  winning  hearts  and  minds,  resilience  to  cyber  attack,  and 
targeting  impact.  In  the  first  case,  winning  hearts  and  minds,  analysts  used  CONSTRUCT  to  assess  a 
diffusion  question:  specifically,  how  to  best  communicate  with  a  population  to  influence  overall 
sentiment  toward  the  United  States  (Louie  &  Carley,  2008).  The  analysts  ran  a  virtual  experiment 
that  considered  messages  via  different  media  at  different  frequencies,  with  messages  coming  from 
both  friendly  and  adversarial  agents.  The  study  compared  three  modes  of  communication:  having  a 
single  speaker  tour  the  area  giving  talks,  publishing  articles  in  a  newspaper,  and  posting  articles  on 
the  web.  Figure  19  shows  the  result  of  this  experiment.  Although  negative  sentiment  toward  the 
United  States  persisted,  the  greatest  mitigation  resulted  from  providing  information  on  the  web, 
even  when  literacy  is  low,  thanks  to  the  high  level  of  access  and  the  ease  of  forwarding  the 
material. 
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Figure  19.  Winning  the  hearts  and  minds. 


In  the  second  case,  analysts  used  CONSTRUCT  to  assess  a  cyber  security  question:  specifically, 
whether  coordination  and  data  sharing  between  combatant  commands  makes  them  more  resilient 
to  cyber  attacks  (Lanham,  Morgan,  &  Carley,  2011).  This  involved  a  virtual  experiment  for  two 
scenarios.  In  the  first  case,  the  commands  did  not  share  data;  in  the  second  case,  they  shared  data. 
Figure  20  shows  that  data  sharing  actually  made  the  commands  less  resilient  in  the  face  of 
reliability  attacks  and  more  resilient  to  integrity  attacks  due  to  the  implicit  "backup  and 
propagation"  of  viruses  and  good,  non-corrupted  information,  respectively. 
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Figure  20.  Resilience  to  cyber  attack. 


In  the  third  example,  analysts  used  CONSTRUCT  for  COA  assessment.  The  specific  example  comes 
from  targeting,  asking  whether  the  expected  performance  of  a  covert  group  degrades  if  the  key 
leader  is  removed  (Carley,  Dombroski,  Tsvetovat,  Reminga,  &  Kamneva,  2003).  As  shown  in  Figure 
21,  removal  of  the  leader  degrades  the  performance  of  this  particular  group;  however,  this  result 
depends  strongly  on  the  network  structure. 


With  Without 
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Figure  21.  Targeting  impact. 


9.3.  Overarching  Comments  on  CONSTRUCT 

CONSTRUCT  is  a  powerful  agent-based  dynamic-network  simulation  engine.  It  can  be  instantiated 
using  actual  or  theoretical  networks.  With  CONSTRUCT,  a  number  of  questions  can  be  addressed 
concerning  the  impact  of  various  interventions  in  the  social-cyber  domain.  Questions  about  the 
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diffusion  of  ideas,  change  in  sentiment,  and  related  changes  in  activities  under  diverse  sociocultural 
and  media  scenarios  can  be  addressed. 

CONSTRUCT  is  illustrative  of  the  type  of  model  in  which  the  social  network  constrains  and  enables 
individual  behavior.  It  is  distinct  from  most  agent-based  models  as  most  such  models  either  do  not 
consider  the  social  network,  or  treat  the  social  network  as  fixed.  It  is  distinct  from  the  generative 
models  as  its  focus  is  to  explain  the  evolution  of  real  social  behavior,  not  to  provide  a  mathematical 
description  of  the  emergence  of  a  network  form  based  on  simple  principles. 

A  key  limitation  is  that  knowledge  is  represented  as  a  binary  string  and  the  amount  of  knowledge 
per  actor  can  be  difficult  to  estimate.  A  second  limitation  is  that  the  time  required  to  instantiate  a 
new  model,  particularly  when  there  are  many  courses  of  action  to  estimate,  can  be  substantial. 

As  we  look  to  the  future,  several  advances  with  CONSTRUCT  are  underway.  First,  CONSTRUCT  is 
being  made  interoperable  with  ORA  so  that  the  analyst  can  automatically  generate  an  input  deck 
for  CONSTRUCT  using  the  network  data  as  input.  This  facilitates  re-use.  Second,  additional  types  of 
media  are  being  added  so  that  CONSTRUCT  can  be  used  to  explore  how  best  to  communicate  with 
populations  to  effect  changes  in  beliefs  using  social  media. 

Finally,  a  key  value  of  CONSTRUCT,  unlike  other  agent-based  models  of  information  diffusion  and 
belief  dispersion,  is  that  CONSTRUCT  takes  into  account  the  social  network  and  allows  it  to  co¬ 
evolve  with  the  culture  over  time.  As  such,  CONSTRUCT  is  valuable  for  general  prediction  of  change 
in  atmospherics.  The  movement  to  enable  CONSTRUCT  to  be  auto-instantiated  from  any  network 
data  in  CONSTRUCT  will  greatly  ease  the  analyst's  ability  to  do  predictive  forecasting. 


10.  Judgmental  Forecasting 

Decision  makers  in  many  fields  rely  on  predictive  systems  to  inform  their  judgments  and  planning 
activities.  Many  of  these  systems  involve  statistical  models  or  algorithms  to  derive  forecasts  from 
relevant  data  sources  (Box,  Jenkins,  &  Reinsel,  2008).  Examples  include  many  econometric  models, 
such  as  those  used  to  forecast  unemployment  rates  or  consumer  price  values  (Makridakis, 
Wheelwright,  &  Hyndman,  1997).  In  other  cases,  statistical  models  may  produce  inputs  to  inform  a 
forecast,  with  a  human  expert  adjusting  the  prediction  based  on  exogenous  factors.  For  example, 
Silver  (2012)  has  recently  demonstrated  methods  for  identifying  and  pooling  information  from 
multiple  data  sources  to  produce  forecasts. 

In  cases  where  no  robust  statistical  models  or  data  streams  are  readily  available,  policy  makers 
frequently  turn  to  the  judgmental  forecasts  offered  by  individual  experts.  However,  a  growing  body 
of  research  indicates  that  the  predictions  of  individual  subject  matter  experts  tend  to  have  limited 
accuracy,  often  failing  to  exceed  the  accuracy  of  non-expert  judges  or  simple  algorithms  (Sanders  & 
Manrodt,  2003;  Tetlock,  2006).  Fortunately,  researchers  are  now  actively  developing  methods  for 
combining  multiple  expert  judgments,  and  these  methods  show  considerable  promise  for 
remedying  the  weaknesses  of  expert  forecasts  (e.g.,  Abramowicz,  2007;  Surowiecki,  2004).  This 
section  describes  issues  and  advances  in  this  emerging  field  of  aggregative  judgmental  forecasting. 
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The  methodological  challenges  for  using  expert  judgments  from  multiple  forecasters  include 
identifying  forecasters  to  survey,  determining  reliable  criteria  for  identifying  good  forecasters,  and 
finding  the  best  ways  to  elicit  information  from  the  experts.  Thereafter,  automated  methods  must 
merge  or  aggregate  the  judgments  provided  by  multiple  forecasters  to  produce  a  "best"  overall 
forecast  and  communicate  the  results  to  the  decision  maker. 

To  illustrate  these  concepts,  consider  the  following  forecasting  problem  posed  at  the  end  of  2011: 
"Will  €1  Euro  buy  less  than  $1.20  U.S.  dollars  at  any  point  before  1  January  2013?"  During  the  last 
quarter  of  2012,  the  Euro  was  trading  at  about  $1.30,  with  some  fluctuations  (Figure  22).  The 
European  debt  crisis,  including  the  poor  financial  health  of  countries  such  as  Greece,  raised  serious 
concerns  about  the  value  of  the  Euro. 


Figure  22.  Daily  value  of  the  Euro  in  U.S.  dollars  in  late  2012.  Adapted  with  permission  from  OANDA 
(2013). 


A  pool  of  forecasters  provided  judgments  for  this  problem.  Responses  from  these  participants  were 
collected  via  a  web-based  interface  in  which  each  respondent  answered  a  small  number  of 
questions,  including  an  assessment  about  whether  the  event  would  happen  (Yes  or  No),  a 
probability  assessment  of  the  likelihood  of  the  event,  and  a  few  questions  about  self-assessment  of 
expertise  and  access  to  knowledge  about  the  forecasting  problem.  Researchers  have  long 
understood  that  aggregate  estimates  built  from  the  individual  opinions  of  a  large  group  of  people 
often  outperform  the  estimates  of  individual  experts  (Surowiecki,  2004).  Therefore,  this  research 
used  a  simple  method  for  aggregating  the  individual  forecasts:  the  average  of  the  probability 
forecasts,  also  known  as  the  unweighted  linear  opinion  pool  (UlinOP).  Figure  23  shows  the  UlinOP 
for  the  Euro  value  forecasting  problem. 
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Figure  23.  Un-weighted  linear  opinion  pool  for  the  Euro  problem.  Adapted  with  permission  from 
OANDA  (2013). 


Given  the  volatility  of  the  value  of  the  Euro,  early  in  the  life  of  this  forecasting  problem  researchers 
might  have  expected  the  value  to  dip  below  $1.20.  However,  as  the  end  date  approached  the 
prospect  seemed  much  less  likely.  The  UlinOP  responded  slowly  to  the  approaching  deadline, 
primarily  because  it  weighted  all  opinions  equally,  including  the  less  informed  ones. 

Improving  this  situation  poses  a  major  challenge  for  judgmental  forecasting.  One  promising 
variation  is  the  prediction  market  (Graefe  &  Armstrong,  2011),  which  accepts  trades  or  bets  on 
accounts  (using  actual  or  "play"  currency),  and  in  so  doing  incentivizes  participants  to  update  their 
forecasts,  particularly  when  a  deadline  is  approaching.  Evidence  suggests  that  prediction  markets, 
especially  using  the  newest  techniques,  are  at  least  as  accurate  as  other  institutions  predicting  the 
same  events  with  a  similar  pool  of  participants  (Berea,  Maxwell,  &  Twardy,  2012;  Nagar  &  Malone, 
2012). 

10.1.  Identifying  Good  Forecasters 

Common  sense  would  suggest  that  relevant  subject  area  knowledge  would  lead  to  improved 
forecasting  performance.  Recent  studies  have  shown  that  forecasters  who  correctly  answer 
knowledge-based  questions  related  to  the  problem  perform  only  slightly  better  than  persons  with 
less  knowledge  (Miller,  Forlines,  &  Regan,  2012;  Miller,  Kirlik,  &  Hendren,  2011).  One  approach  to 
identifying  good  forecasters  is  to  explore  psychological  factors,  such  as  education,  demographics, 
and  various  measures  of  aptitude  and  personality.  Several  studies  have  found  a  small  but 
consistent  benefit  from  certain  personality  traits  (Poore,  Regan,  Miller,  Forlines,  &  Irvine,  2012; 
Ungar,  Mellers,  Satopaa,  Tetlock,  &  Baron,  2012).  Retrospective  analysis  reported  by  Poore  et  al. 
(2012)  reveals  performance  distinctions  resulting  from  personality,  deductive  and  inductive 
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approaches  to  problem  solving,  and  different  motivations  for  problem  solving.  By  selecting 
forecasters  with  the  appropriate  aptitude  and  personality  traits,  researchers  can  realize  a  small  but 
statistically  significant  boost  in  performance. 

10.2.  Aggregation  of  Results 

Developing  better  methods  of  aggregating  the  individual  forecasts  to  achieve  greater  accuracy 
presents  an  interesting  challenge  (Rantilla  &  Budescu,  1999).  Recent  research  has  shown  that 
combining  judgments  through  averaging  leads  to  poor  prediction  performance,  as  illustrated 
above.  One  intuitive  approach  is  to  assign  greater  weight  to  predictions  made  by  "experts,"  but,  as 
noted,  identifying  the  experts  among  the  pool  of  forecasters  is  not  easy.  Ways  to  address  this 
problem  include  eliciting  additional  information  to  assess  each  forecaster's  expertise,  sharing 
information  in  a  way  that  increases  expertise  among  the  forecasters,  and  training  forecasters  in 
skills  that  enhance  performance. 

To  illustrate  the  effects  of  various  factors  on  the  accuracy  of  the  forecast,  we  return  to  the  Euro 
example.  Recall  that  the  event  did  not  occur,  so  lower  probabilities  are  more  accurate.  As  we  saw 
earlier,  the  simple  average  or  UlinOP  does  not  perform  very  well.  A  simple  procedure  that  assigns 
greater  weight  to  the  most  recent  forecasts,  with  some  smoothing,  shows  better  performance  as 
the  end  date  approaches.  Forecasters  who  make  their  prediction  closer  to  the  end  date  have  the 
advantage  of  more  current  information  and,  consequently,  provide  better  forecasts. 

When  participants  in  the  forecasting  pool  can  update  their  forecasts,  a  news  report  indicating  a 
change  in  conditions  related  to  the  forecasting  problem  may  trigger  adjustments.  In  Forlines  et  al. 
(2012),  relatively  few  participants  actually  provided  updates.  A  heuristic  aggregation  procedure 
weighted  the  respondents  by  update  frequency,  propensity  to  make  estimates  at  the  top  or 
bottom  of  the  probability  range,  and  access  to  knowledge.  If  the  number  of  updates  to  the  forecast 
functions  as  a  surrogate  for  the  respondent's  level  of  interest  or  engagement,  forecasters  who 
update  frequently  are  likely  to  be  more  accurate.  Furthermore,  updating  means  that  the  latest 
forecast  is  presumably  based  on  the  latest  information.  The  probability  estimate  can  be  seen  as  an 
indication  of  confidence,  so  greater  weight  was  given  to  more  extreme  probability  estimates,  i.e., 
values  closer  to  zero  or  one.  Questions  about  access  to  knowledge  yield  an  implicit  self-assessment 
of  the  respondent's  expertise.  The  study  weighted  the  individual  forecasts  by  these  three  factors 
and  combined  them  to  produce  the  heuristic  forecast  depicted  in  Figure  24.  This  aggregation 
scheme  dramatically  outperformed  the  UlinOP  in  this  case,  although  some  volatility  is  evident  early 
on.  This  technique  has  also  performed  well  on  a  number  of  other  forecasting  problems,  although  it 
has  failed  as  well  (Forlines,  Miller,  Prakash,  &  Irvine,  2012). 
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Figure  24.  Comparison  of  selected  aggregation  methods.  Adapted  with  permission  from  Forlines  et 
al.  (2012). 


Current  research  is  developing  new  aggregation  methods  that  may  well  outperform  the  simple 
average  in  most  cases.  In  addition,  active  research  explores  ways  to  identify  the  best  forecasters 
(Prelec,  Seung,  &  McCoy,  2012)  and  to  improve  methods  for  aggregating  the  information  (Irvine, 
Srinivasamurthy,  Regan,  &  Prelec,  2012). 

10.3.  Lessons  Learned  and  New  Directions 

Realizing  good  performance  from  judgmental  forecasts  poses  many  challenges  (Lawrence, 
Goodwin,  O'Connor,  &  Onkald,  2006;  Tetlock,  2006).  In  many  cases,  seemingly  less  qualified 
individuals  can  perform  almost  as  well  as  experts  by  conducting  a  moderate  amount  of  research. 
Some  important  lessons  from  recent  studies  show  that  training  can  help  forecasters  think  more 
clearly  about  the  forecasting  problem,  and  that  on  average  trained  forecasters  perform  better 
(Ungar  et  al.,  2012).  Good  elicitation  techniques  can  also  improve  forecasting  performance 
(Deloatch,  Marmarchi,  &  Kirlik,  2013;  Marmarchi,  Deloatch,  &  Kirlik,  2013;  Miller  et  al.,  2011;  Tsai, 
Miller,  &  Kirlik,  2011).  Aggregation  of  expert  judgments  outperforms  individual  forecasts  (Graefe  & 
Armstrong,  2011),  and  several  methods  outperform  the  UlinOP  under  a  wide  range  of  conditions 
(Forlines  et  al.,  2012;  Irvine  et  al.,  2012;  Ungar  et  al.,  2012;  Miller,  Forlines  &  Irvine,  2013). 

One  potential  avenue  for  improving  forecasts  involves  the  development  of  better  hybrid  methods 
that  combine  the  best  elements  of  model-based  (statistical)  forecasts  with  expert  judgment. 
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Returning  to  the  forecasting  problem  related  to  the  Euro  exchange  rate,  the  underlying  data  are 
amenable  to  statistical  modeling.  The  autocorrelation  function,  which  measures  the  correlation 
between  observations  separated  by  a  fixed  difference  in  time,  is  high  for  small  differences  in  time 
and  tapers  off  as  the  time  interval  increases.  Thus,  a  statistical  model  that  predicts  future  values 
based  on  the  current  value  and  the  recent  past  will  probably  perform  well  over  a  short  period,  but 
become  progressively  less  accurate  over  a  longer  time  horizon.  Some  researchers  (e.g.,  Nagar  & 
Malone,  2011)  have  already  begun  to  explore  the  potential  of  human-model  hybrids  that  use  a 
weighted  combination  of  model-based  forecasts  and  expert  judgments,  where  the  weights  depend 
on  the  time  horizon. 
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11  Making  sense  of  Social  Radar:  V-SAFT  as  an  intelligent 
machine1 

Ian  S.  Lustick,  Lustick  Consulting/University  of  Pennsylvania 


1.  Introduction 

"Social  radar"  is  a  metaphor:  in  other  words,  a  model.  A  model  projects  something  known  about  a 
"source"  onto  something  less  well  known— a  target.  We  see  the  moon  (the  target)  differently  when 
we  hear  the  metaphor:  "The  moon  is  a  ghostly  galleon."  In  effect,  the  poet  asks  the  reader  to  use 
"ghostly  galleon"  as  a  model  for  the  moon.  How  we  visualize  the  moon  when  we  use  that  model 
results  from  our  individual  expectations,  emotions,  and  images  associated  with  the  idea  of  a 
"ghostly  galleon."  Under  the  influence  of  the  metaphor,  i.e.,  when  employing  "ghostly  galleon"  as  a 
model  for  the  moon,  we  see  the  moon  not  as  an  astronomical  object  but  as  a  spooky  vision, 
floating  silently,  mysteriously,  perhaps  ominously,  through  the  clouded  darkness.2  Taking  "social 
radar"  seriously  as  a  metaphor,  and  therefore  as  a  model,  can  illuminate  many  aspects  of  this 
vision  for  forecasting. 


2.  Objective 

Radar  detects  and  tracks  objects  in  the  sky  by  sending  signals  that  bounce  back  in  decipherable 
patterns  to  reveal  the  position,  size,  direction,  and  velocity  of  air-  or  spaceborne  objects.  Sonar 
signals  reflected  from  the  rocks,  canyons,  and  other  formations  on  the  ocean  bottom  depict  an 
entire  landscape  of  shapes,  with  fish  or  ships  discernible  by  their  shape  and  especially  their 
movement.  Actively  and  systematically  interrogating  the  social  environment  for  "bounce-back" 
signals  about  occurrences  represents  a  simple  analogy  with  straightforward  implications. 

Because  the  laws  of  physics  are  fairly  well  understood,  projecting  the  path  of  objects  whose 
signature  has  been  detected  via  radar  or  sonar  is  rather  straight-forward,  at  least  when  it  comes  to 
assessing  probable  immediate  trajectories.  But  the  dynamics  of  the  social  realm  are  much  more 
complex  than  those  normally  faced  by  radar  engineers.  Moreover  our  theories  of  "social  physics" 
are  much  weaker.  To  some  extent,  extrapolation  of  present  trends  is  possible  and  useful,  enabling 
an  uncomplicated  analogy  between  tracking  and  extrapolating  the  path  of  a  moving  object  and 


1  Approved  for  Public  Release;  Distribution  Unlimited.  Case  Number  14-2487 
This  work  was  supported  by  Department  of  Defense  Contract  N00014-10-C-0042 
Copyright  ©  2014  The  MITRE  Corporation. 

2  For  detailed  consideration  of  the  warrants  for  and  limits  to  thinking  of  models  as  metaphors  and  vice  versa 
see  Weisberg  (2013). 
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tracking  and  extrapolating  patterns  of  human  sentiment  or  behavior  (Maybury,  2010).  But 
forecasting  probabilities  for  most  kinds  of  political,  social,  cultural,  and  economic  events  of  interest 
for  policy  makers  and  analysts  requires  much  more  than  extrapolation.  Thinking  deeply  about 
"social  radar"  as  a  metaphor  and  therefore  as  a  model  can  help  us  appreciate  the  challenge  of 
forecasting  in  the  human  social,  cultural,  and  behavioral  sphere,  and  guide  us  toward  appropriate 
strategies  for  addressing  that  challenge.  The  main  problem  is  that  the  future,  by  definition,  does 
not  exist,  so  signals,  or  questions  addressed  to  it  cannot  be  reflected  back,  or  answered.  There  is 
simply  nothing  "there"  to  reflect  our  signals  or  to  form  the  basis  of  an  answer  to  our  queries. 

To  make  sense  of  "social  radar"  as  a  model  for  investigating  the  future,  i.e.,  for  forecasting,  we 
must  do  three  things:  (1)  produce  a  surrogate  for  the  future  that  can  serve  as  the  "target",  (2) 
develop  techniques  for  interrogating  that  target— signals  we  direct  toward  it  in  the  form  of  queries 
that  return  analyzable  patterns  of  information,  and  (3)  design  and  implement  visualizations  to 
transform,  filter,  and  present  data  in  accessible  and  interactive  forms. 


3.  Approach 

One  way  to  produce  a  target  of  something  that  does  not  exist— such  as  the  future— is  to  virtualize 
it.  The  well-known  and  well-established  technique  of  scenario  analysis  rests  on  this  understanding. 
Instead  of  loosely  pondering  the  vast  number  of  discrete  events  that  could  occur,  an  analyst  or 
group  of  analysts  devises  a  small  set  of  stories  about  how  the  present  might  evolve  toward  the 
future  with  respect  to  a  particular  issue  area  or  a  particular  outcome  by  following  an  imagined 
"punctuation"  of  prevailing  trends.  This  technique  creates  surrogate  futures  as  objects  of  analysis. 
These  scenarios  are  "virtual"  in  that  they  exist,  or  at  least  are  translated  from,  images  in  the  minds 
of  their  producers.  Such  translation  involves  an  arduous  process. 

The  future  has  at  least  as  many  dimensions  as  the  present.  "Transition  rules"  (i.e.,  the  laws  of  social 
behavior  at  all  levels  of  analysis)  are  unknown,  or  at  least  only  partially  understood.  Hence,  every 
story  about  what  the  future  may  hold  must  include  large  doses  of  arbitrary,  even  idiosyncratic, 
decisions  about  what  dimensions  and  variables  will  dominate,  in  what  combinations,  in  what 
sequence,  and  with  what  consequences.  This  reality  turns  scenario  production  into  more  of  an  art 
than  a  science,  posing  difficult  challenges  for  traditional  techniques  of  scenario  production  for  the 
shaping  of  the  surrogate  future  to  be  interrogated.  In  particular,  four  factors  play  crucial,  non- 
scientifically  justified  roles  in  typical  processes  of  scenario  production:  (1)  employment  of  seductive 
but  highly  arbitrary  "black  swan"  details,  (2)  a  vast  array  of  human  heuristics  or  "psychologies"  that 
tend  to  trump  scientifically  based  inferences,  (3)  standard  procedures  for  the  entertainment  of 
three  options  (two  extremes  and  a  "mixed"  or  "intermediate"  version),  and  (4)  political  biases  or 
political  influences  on  the  analytic  process. 

If  we  cannot  rely  on  our  imagination  to  produce  scenarios  as  a  suitable  surrogate  for  the  future, 
toward  what  are  we  to  point  our  "social  radar"  for  forecasting  purposes?  After  all,  to  make  sensible 
the  metaphor/model  of  "social  radar"  directed  at  the  future,  we  must  direct  our  questions  (the 
outgoing  radar  signals)  toward  a  target.  The  real  target  is  the  future,  but  the  future,  by  definition, 
does  not  (yet)  exist.  Accordingly,  we  need  a  surrogate  for  the  future  as  a  target  for  our  "social 


Making  sense  of  social  radar:  V-SAFT  as  an  intelligent  machine  |  319 


radar"  signals.  Our  forecasts  can  be  no  better  than  the  surrogate  we  use  for  the  way  the  future 
could  emerge.  As  noted,  a  short  list  of  BOGSAT  ("Bunch  of  Guys  Sitting  Around  Talking")-generated 
scenarios  is  clearly  inadequate.  The  target  toward  which  forecast-oriented  "social  radar"  directs  its 
queries  must  instead  be  produced  in  a  logically  consistent,  theoretically  coherent,  and  empirically 
supportable  manner.  Instead  of  bouncing  our  "signals"  off  a  small  number  of  idiosyncratic  and 
perhaps  wholly  impossible  scenarios,  we  must  imagine  the  future  as  a  high-dimensional  space  of 
possibilities.  The  bounce-back  we  receive  from  this  space  in  response  to  our  queries  will  then 
enable  a  mapping  not  of  the  actual  future,  but  of  the  shapes  and  contours  of  that  space  of 
possibilities. 

Let  us  take  a  closer  look  at  the  distinction  between  the  "actual"  future  and  the  space  of  possible 
futures.  From  any  time  labeled  the  "present,"  we  can  only  imagine  or  visualize  the  "future" 
correctly  as  a  large  set  of  multidimensional  trajectories.  These  trajectories  trace  through  an 
enormous,  but  not  boundless  or  totally  disorganized,  "state  space"  of  the  possible  worlds  that 
could  evolve  from  the  world  that  we  experience  as  the  present.  In  other  words,  looking  forward 
from  the  present,  we  can  imagine  many  ways,  along  many  dimensions,  some  important  and  some 
not,  in  which  the  actual  set  of  events  of  our  world  can  vary. 

Most  traditional  approaches  to  forecasting  seek  to  anticipate  an  "actual"  future  or  its  probability, 
at  least  along  some  specific  dimension.  Implicit  in  the  view  that  we  could  predict  the  trajectory  of 
the  actual  future  in  some  detail  is  either  a  faith-based  belief  in  prophecy  or  an  utterly  mistaken 
assessment  of  the  power,  coherence,  and  comprehensiveness  of  available  social  theory.  To  make 
matters  worse,  a  salient  but,  in  any  particular  case,  unanticipatable  fraction  of  the  explanation  of 
what  actually  happens  will  emanate  from  causes  located  below  the  analytic  horizon  of  any  of  our 
theories. 

With  these  considerations  in  mind,  we  must  instead  understand  the  process  of  forecasting  political 
events  of  interest  (EOls)  in  a  particular  setting  as  an  exercise  in  tracing  the  contours  of  the  space  of 
possible  futures,  along  relevant  dimensions,  rather  than  searching  for  the  one  actual  trajectory  that 
our  world  will  follow  through  that  space.  "Point  predictions"  are  fool's  gold.  For  complex  social 
problems  systematic  analysis  cannot  produce  them  and  attempts  to  do  so  encourage  a 
fundamentally  misleading  construction  of  the  problem.  What  we  can  achieve  are  informed 
judgments  about  probabilities  and  associations  within  distributions  of  outcomes.  Figure  1 
highlights  the  irrationality  of  seeking  to  predict  the  actual  future  by  locating  that  future  as  a  nearly 
imperceptible  dot  in  a  schematic  depiction  of  intersecting  spaces  of  the  conceivable,  the  possible, 
the  plausible,  and  the  probable. 
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Never  Conceived  of  or  Inconceivable 


Figure  1.  Categories  of  worlds.  The  dot  indicates  the  actual  future,  with  all  possible  outcomes 
depicted. 


The  diagram  merits  some  consideration.  It  depicts  all  possible  outcomes  as  a  subset  of  all  those 
that  are  conceivable;  all  plausible  outcomes  as  a  subset  of  those  that  are  possible;  all  probable  (or 
not  improbable)  outcomes  as  a  subset  of  those  that  are  plausible;  and  the  actual  outcome  as 
located  within  the  realm  of  the  probable.  Note  that  except  for  the  dot  representing  the  actual,  all 
other  descriptions  of  the  future  world  in  this  diagram,  be  they  probable,  plausible,  possible, 
conceivable,  or  never  conceived  of,  can  be  classified  as  "counterfactuals"— accounts  contrary,  or 
counter,  to  what  the  world  actually  was,  or  what  it  became. 

It  is  important  to  note  that  although  people  might  like  to  imagine  that  the  landscape  of  the  future 
is  as  orderly  as  this  depiction  suggests,  that  landscape  is  in  fact  likely  to  be  considerably  messier 
and  more  complex.  For  example,  the  ratio  of  the  possible  to  the  conceivable  may  be  substantially 
greater  than  we  think.  Some  of  the  outcomes  we  might  consider  probable  may  in  fact  be  not  only 
implausible,  but  downright  impossible.  The  "actual"  world  that  emerges  may  be  one  of  the  variants 
not  considered  "probable,"  but  only  "plausible."  Even  some  of  those  possibilities  we  consider 
relatively  probable  may  be  merely  possible,  or  actually  impossible.  For  illustrative  purposes,  Figure 
2  incorporates  these  potential  misconceptions. 


Figure  2.  Categories  of  worlds  if  imagined  incorrectly.  The  dot  indicates  the  actual  future,  with 
possible  outcomes  depicted  with  real-world  messiness. 
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Despite  the  difficulty  of  producing  a  reasonably  accurate  surrogate  for  the  contours  of  what  should 
be  considered  "possible/'  precisely  that  surrogate  is  required  for  any  forecasting  tool  to  be 
consistent  with  the  idea  of  "social  radar."  Whatever  technique  modelers  use  to  accomplish  this 
task  with  respect  to  the  future  of  a  complex  society,  it  must  be  governed  by  principles  or  be  based 
on  assumptions  that  enable  what  is  produced  to  be  treated  as  not  only  logically  but  also  empirically 
possible.  The  technique  must  also  be  capable,  at  very  low  marginal  cost,  of  generating  large 
numbers  of  empirically  realizable  multidimensional  trajectories  or  chronologies. 


4.  Results 

The  balance  of  this  paper  uses  the  Virtual  Strategic  and  Forecasting  Tool  (V-SAFT)  as  an  example  to 
explain  how  a  "social  radar"  model  could  satisfy  these  requirements.3  The  discussion  then  employs 
the  state  space  of  the  future  produced  by  V-SAFT  as  a  target  for  queries  designed  to  distinguish 
plausible  and  probable  futures  from  those  that  are  merely  possible.  It  also  covers  how  the  tool  can 
monitor  change  across  regularly  updated  versions  of  that  state  space  in  order  to  generate 
indications  and  warnings,  and  can  offer  opportunities  for  what-if  experiments  to  help  adjudicate 
disputes  over  the  implications  of  different  assessments  of  key  variables  in  the  real  world  or  to 
conduct  what-if  experiments. 

4.1.  V-SAFT  Architecture  and  Processes 

As  displayed  in  Figure  3,  V-SAFT  incorporates  model-building,  experimentation,  and  analysis.  Most 
processes  are  automated  or  semi-automated.  In  the  left-hand  column,  under  model  building,  the 
tool  transfers  data  on  geographical  and  administrative  districts  in  a  country  to  a  pixelated 
computer  grid  in  a  form  that  can  be  integrated  within  the  frame  of  a  PS-1  model.  PS-1  is  a  multi-use, 
agent-based  modeling  platform  developed  by  Lustick  Consulting  for  building  and  employing 
complex  virtualization  models  employed  in  forecasting  and  analysis  based  on  cellular  automata. 


3  V-SAFT  technology,  developed  by  Lustick  Consulting,  is  a  stand-alone  offshoot  of  work  done  within  the 
Worldwide  Integrated  Crisis  Early  Warning  System  (W-ICEWS)  team  at  Lockheed  Martin  Advanced  Technology 
Laboratories,  a  project  funded  by  the  Department  of  Defense's  Fluman  Social  Culture  Behavior  Modeling 
program. 
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Figure  3.  V-SAFT  architecture  and  processes. 


A  typical  country  model  can  involve  between  1,000  and  4,500  agents.  As  indicated  at  the  top  of  the 
left-hand  column  in  Figure  3,  model  building  begins  by  mapping  the  target  country  (or  sub-region) 
into  PS-1.  That  landscape  is  then  loaded  into  the  Generic  Political  Model  (GPM).  The  GPM 
condenses  and  federates  a  variety  of  political  science  theories  about  how  states  and  societies 
operate  and  relate  to  one  another.  Although  no  social  science  theories  have  the  certainty  of  some 
physical  laws,  all  analysts  naturally,  but  unselfconsciously,  use  theories  they  believe  in  to  produce 
judgments  about  the  future  and  conclusions  about  the  likely  effects  of  alternative  assumptions  or 
courses  of  action.  The  GPM  and  other  modules  encode  relatively  well-corroborated,  even 
consensual  theories  developed  by  political  scientists  by  translating  them  into  micro  rules  for  agent 
behavior,  communication,  and  updating  (Garces  &  Alcorn,  2012;  Lustick,  Alcorn,  Garces,  & 
Ruvinsky,  2012).  The  GPM  then  uses  demographic,  cultural,  electoral,  economic,  and  subject 
matter  expert-derived  data,  including  information  on  elite  networks,  to  endow  agents  with 
attributes  that  render  the  model  consistent  with  and  tuned  to  realities  in  the  target  country  at  a 
specified  time.  Those  attributes  include  a  repertoire  of  currently  available  affiliations,  or  identities, 
with  one  among  them  activated,  i.e.,  publicly  on  display,  at  any  point  in  time. 

A  key  module  attached  to  the  GPM  is  the  Dynamic  Political  Hierarchy,  which  establishes  the 
"dominant  identity"  or  dominant  "group"  at  each  succeeding  time  step.  By  then  determining 
patterns  of  overlap  or  isolation  among  various  groups  as  reflected  in  the  identity  activation  of 
agents  and  their  identity  complexions,  the  Dynamic  Political  Hierarchy  sorts  agents  into  key 
political  categories:  dominant,  incumbent,  regime,  system,  and  non-system.  When  agents 
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dissatisfied  or  angered  by  their  circumstances  mobilize  legally,  semi-legally,  or  violently,  they  act. 
The  type  of  action  they  take  depends  on  their  position  within  the  overall  constellation  of  power, 
affiliation,  and  commitment  prevailing  at  that  time  within  the  model  (Lustick  et  al.,  2012). 

In  the  middle  column  of  Figure  3,  "Experimentation/'  V-SAFT  sends  individual  models  forward  in 
time.  After  a  standardized  "burn-in"  period,  each  model  time  step  represents  approximately  one 
week,  with  60  time  steps  taken  as  a  year.  As  a  standard,  V-SAFT  generates  1,000  distinctive 
trajectories.  Two  stochastic  elements  ensure  the  distinctiveness  of  each  trajectory:  (1)  a 
randomized  distribution  of  agent  attributes  at  t- 0  that  produces  a  unique  version  of  the  model 
consistent  with  our  knowledge  of  district-level  affiliations  and  sentiments  and  including  the  density 
and  communication  patterns  of  various  influence  networks,  and  (2)  a  unique,  stochastically 
produced  and  recorded  sequence  of  small  perturbations  in  the  bias  assignments  of  affiliative 
identities  that  is  associated  with  each  model  run.  These  perturbations  act  as  a  source  of  exogenous 
shocks  that  take  place  below  the  analytic  horizon.  V-SAFT  sets  the  size  and  regularity  of  these 
shocks  within  a  standardized  range  across  all  models,  but  can  be  adjusted  for  political 
environments  judged  as  unusually  calm  or  turbulent. 

The  developers  ran  elaborate  sensitivity  studies  that  involved  measuring  variance,  means,  and 
kurtosis  across  sample  sizes  up  to  10,000.  The  findings  strongly  suggested  that  more  than  1,000 
runs  per  model  condition  would  yield  sharply  diminishing  returns  in  terms  of  producing  interesting 
variations. 

For  each  country  model  run  with  V-SAFT,  we  view  the  1,000  trajectories  as  chronicles  depicting  the 
state  space  of  the  ways  in  which  events  could  unfold  during  the  coming  year  in  the  model's  target 
country.  In  effect,  then,  we  treat  this  set  of  trajectories  as  if  it  were  the  "possible"  futures 
consistent  with  our  best  knowledge  of  the  present,  as  condensed  into  the  model.  In  precise  terms, 
the  trajectories  represent  the  set  of  futures  for  which  our  model  has  provided  "existence  proofs"  — 
proofs  that  the  sequences  of  events  tracked  in  these  futures  could  be  realized,  assuming  the 
correctness  of  the  model  and  the  reliability  of  the  computer's  implementation  of  the  model's 
routines. 

Every  month  we  update  each  model  with  significant  new  sources  of  data  (elections,  censuses, 
subject  matter  expert  reports,  etc.)  that  become  available.  Models  are  also  updated  by  an 
automated  process  that  compares  the  first  month  of  each  new  trajectory  of  a  possible  future  to 
information  about  protests  or  violence  that  actually  occurred  in  the  country  under  study  during  the 
previous  month  to  eliminate  the  subset  of  futures  that  can  no  longer  be  considered  consistent  with 
the  country's  "present."  We  run  the  model  until  we  have  collected  a  set  of  1,000  trajectories,  each 
consistent  with  the  country's  general  condition  at  t= 0. 

Having  thus  produced  the  target  for  our  "social  radar"  signals,  we  move  to  the  third  column  in 
Figure  3:  analysis.  We  address  standardized  queries  about  events,  patterns,  and  outcomes  of 
interest  to  the  data  base  containing  information  about  all  1,000  trajectories.  Answers  to  these 
queries  are  equivalent  to  the  bounce-back  signal  received  by  a  radar  installation  that  indicates  the 
location,  shape,  direction  of  movement,  and  velocity  of  the  target. 
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Displays  of  the  data  that  emerge  from  these  queries  communicate  a  great  deal  about  the  contours 
of  the  state  space  of  the  future  as  suggested  by  the  behavior  of  the  model.  V-SAFT  then  uses 
features  of  this  state  space  to  generate  forecasts  that  distinguish  among  probable,  plausible,  or 
merely  possible  outcomes.  The  aim,  again,  is  not  to  make  "point  predictions"  of  the  "actual"  future 
in  any  detail,  but  to  give  users  systematic,  quantifiable,  and  time-sensitive  indications  and  warnings 
about  EOls.  Since  the  data  are  recorded  within  specific  trajectories,  we  can  identify  the  conditions 
that  tend  to  produce  desirable  or  undesirable  outcomes,  whether  relatively  common  or  extremely 
rare,  and  use  them  to  design  specific  and  cost-effective  mitigation  strategies. 

4.2.  Queries 

With  respect  to  the  "social  radar"  model  of  forecasting,  we  have  noted  that  the  "target"  of  the 
signals  emitted  by  the  radar  instrument  is  the  simulated  state  space  of  the  possible  for  the  country, 
region,  or  population  under  study.  But  what  are  the  signals?  "Social  radar"  must  send  standardized 
signals  toward  targets  of  interest  to  gain  information  from  the  distinctiveness  of  the  patterned 
reflection  of  those  signals.  Radar  operators  identify  different  types  of  objects,  birds,  fighter  aircraft, 
bombers,  helicopters,  etc.,  by  their  characteristic  signatures.  Similarly,  V-SAFT  operators  use 
queries  directed  toward  the  data  resulting  from  the  production  of  1,000  simulated  futures. 
Responses  to  these  queries  reflect  the  contours  of  the  state  space  of  the  country's  possible  futures. 
Visualizations  of  the  answers  to  these  queries  are  the  equivalent  of  visual  images  on  a  radar  scope 
and  can  be  interpreted  in  standard  ways  to  conduct  analysis  of  the  target  and  the  direction  and 
velocity  of  events. 

4.3.  Forecasts 

Each  month  human  analysts  draft  a  summary  forecast  for  each  country  model,  based  on  a  series  of 
visualizations  focused  on  a  variety  of  dimensions.  We  use  the  summary  forecast  for  Bangladesh  for 
the  year  March  2013-February  2014  to  illustrate  the  production  of  these  forecasts  from  data 
displayed  in  these  standardized  visualizations.  Unless  otherwise  noted,  all  visuals  are  available  on 
the  web  for  inspection  and  drill-down,  as  are  operational  definitions  of  terms. 

Here  is  the  forecast: 

V-SAFT' s  Bangladesh  model  forecasts  a  somewhat  more  than  75%  probability  of  a  sustained 
domestic  political  crisis  between  March  2013  through  February  2014  and  only  a  10% 
probability  of  avoiding  any  sort  of  domestic  political  crisis  during  this  period.  Insurgent  activity 
and  incidents  of  ethno-religious  conflict  have  a  roughly  75%  probability  of  appearing  in  this 
period,  but  the  likelihood  of  these  activities  sustaining  themselves  for  three  months  or  more 
is  very  low  (below  3%).  These  probabilities  remain  more  or  less  the  same  across  the  entire 
forecast  period.  Nor  do  probabilities  vary  widely  depending  on  the  complexion  of  the 
governing  coalition.  Under  a  broad-based  nationalist  government,  a  business  dominated 
coalition,  or  one  based  on  Bengali  appeals  that  is  yet  not  dominated  by  the  Awami  League, 
Bangladesh  has  the  best  chance  of  enjoying  a  year  relatively  free  from  instability  or  severe 
instability.  But  the  model  shows  a  wide  variety  of  different  coalitions  capable  of  emerging, 
most  of  which  are  associated  with  some  significant  instability  or  severe  instability. 
Probabilities  for  at  least  one  of  these  coalitions  range  from  a  low  of  15%  in  governments 
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dominated  by  non-partisan  nationalists  or  business  elites,  to  a  high  of  nearly  65%  under 
governments  oriented  toward  narrow  civic  and  technocratic  appeals.  Compared  to  last 
month's  forecasts,  a  Muslim  dominated  government  is  no  longer  evaluated  as  plausible,  while 
a  business  dominated  government  is  to  be  considered  plausible,  and  not  merely  possible.  V- 
SAFT's  overall  assessment  for  the  coming  year  is  that  governing  elites  in  Bangladesh  will  likely 
face  higher  than  average  amounts  of  illegal  and  violent  political  mobilization  over  the  coming 
year.  This  represents  a  substantial  trend  across  the  last  five  months  of  V-SAFT  forecasts  for 
Bangladesh.  Keeping  in  mind  what  is  normal  for  Bangladesh,  this  month's  forecast  registers 
not  just  an  increasing  probability  of  instability,  but  an  increasing  probability  of  severe 
instability.  (Lustick  Consulting,  2013) 


The  summary  forecast  begins  with  probabilities  for  key  EOls  of  historical  importance  for  U.S. 
civilian  and  military  planners  concerned  with  Bangladesh.  The  "somewhat  more  than  75%" 
likelihood  of  a  sustained  domestic  political  crisis  (DPC)  over  the  forecast  year  and  the  unlikelihood 
of  Bangladesh  avoiding  any  period  of  domestic  political  crisis  during  that  time  frame  are  read  off 
the  "High  Impact  Possibilities"  chart  (Figure  4). 
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Figure  4.  Probabilities  for  key  EOls  of  historical  importance  in  Bangladesh  such  as  the  75%  chance 
of  a  domestic  political  crisis. 
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The  extended  "occurs"  bar  for  "Domestic  Political  Crisis"  shows  that  90%  of  the  weeks  comprising 
all  1,000  simulated  years  of  possible  Bangladesh  futures  in  the  model  contain  behavior  coded  as 
corresponding  to  the  definition  of  a  DPC:  organized,  mostly  nonviolent  opposition  to  the 
government  that  is  significant  enough  to  threaten  the  integrity  of  the  ruling  coalition.  In  practice 
this  definition  must  render  a  description  of  what  agents  in  the  Bangladesh  model  are  doing  in  a 
language  the  computer  can  understand.  For  illustrative  purposes,  the  operational  definition  of  DPC 
is:  "DPC  exists  in  a  timestep  when  the  aggregate  influence  of  protesting  agents  (protest)  multiplied 
by  the  protest_threshold  (6)  is  greater  than  the  aggregate  influence  of  the  dominant  identity 
(data$dominant_activation)."  The  routine  that  implements  this  category  as  a  rule  for  data 
collection  is:  dpc_exists  =  (protest*protest_threshold>data$dominant_activation).  The  bar  at 
roughly  55%  likelihood  indicates  the  "slightly  more  than  75%"  probability  of  a  sustained  DPC, 
showing  how  large  a  subset  of  those  weeks  were  occupied  by  a  DPC  that  lasted  at  least  three 
consecutive  months. 

The  summary  forecast  also  draws  attention  to  the  likelihood  of  at  least  some  insurgent  activity  or 
ethnic-religious  violence.  However,  in  view  of  the  very  small  size  of  the  red  bars  in  those 
categories,  the  forecast  notes  that  sustained  insurgencies  or  periods  of  ethnic-religious  violence 
occurred  so  rarely  in  this  set  of  trajectories  that  they  should  be  considered  implausible— in  other 
words,  possible,  but  extremely  unlikely. 

How  are  these  probabilities  for  specific  EOls  distributed  over  the  forecast  year?  Figure  5  shows  the 
months  of  the  coming  year  on  the  x-axis  and  proportions  of  futures  in  the  state  space  registering 
EOI  activity  on  the  y-axis.  Thus,  as  noted  in  the  summary  forecast,  we  see  that  "[Tjhese 
probabilities  remain  more  or  less  the  same  across  the  entire  forecast  period,  with  a  slight  trend 
toward  increasing  risk  after  April  2013." 
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Figure  5.  Probabilities  for  key  "events  of  interest"  in  Bangladesh  over  the  forecast  year,  with  DPC 
relatively  stable  at  50%. 
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The  summary  forecast  indicates  that  instability,  including  severe  instability,  is  associated  with  many 
different  constellations  of  political  power  anticipated  as  plausible  within  the  forecast  year.  In  Figure 
6  different  groups  or  political  appeals  that  the  model  identifies  as  plausibly  able  to  play  a  dominant 
and  organizing  role  in  Bangladesh  governing  coalitions  over  this  time  period  are  arrayed  on  the  x- 
axis.  The  width  of  the  bar  associated  with  individual  dominant  groups  or  identities  signifies  the 
proportion  of  weeks  within  all  1,000  trajectories  during  which  they  are  dominant.  The  width  of  the 
vertical  segments  in  each  column  indicates  how  that  amount  of  the  state  space  is  divided  into 
periods  of  "calm"  (green),  "intense  politics"  (blue),  "instability"  (yellow),  and  "severe  instability" 
(red). 

Going  somewhat  beyond  the  explicit  statements  in  the  summary  forecast,  we  can  observe  that 
while  the  most  likely  coalitions— those  dominated  by  nationalist,  Bengali,  or  Awami  League  ideas 
and  elites— hold  out  a  nearly  80%  chance  of  avoiding  instability  or  severe  instability,  no 
government  that  the  model  deems  plausible  has  more  than  a  15%  likelihood  of  enjoying  calm 
throughout  the  year.  A  government  based  on  alliances  led  by  bureaucratic  state  elites  faces  the 
greatest  likelihood  of  both  instability  and  severe  instability. 
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Figure  6.  Dominant  groups  and  levels  of  instability  in  Bangladesh.  Along  the  x-axis  are  different 
groups  identified  by  the  model  as  plausible  anchors  of  ruling  coalitions  in  Bangladesh  in  this  time 
period.  The  width  of  the  columns  associated  with  political  dominance  by  different  groups  signifies 
the  proportion  of  weeks  when  they  are  dominant,  the  width  of  the  bars  within  each  column 
indicates  how  that  amount  of  the  state  space  is  divided  into  periods  of  "calm",  "intense  politics", 
"instability",  and  "severe  instability". 
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By  probing  more  deeply,  we  can  see  which  groups  are  allies,  opponents,  or  radical  opponents  of 
the  government  when  bureaucratic  state  elites  dominate  the  coalition  (Figure  7)  or  when 
"globalizing,  Western-oriented"  elites  dominate  (Figure  8).  We  see  that  these  types  of 
governments,  when  they  have  significant  allies  at  all,  tend  to  rely  on  corrupt  and  criminal  elements 
and  on  the  (highly  corrupt)  business  community.  In  Figures  7-9  and  Figure  11  groups  and  identities 
listed  along  the  x-axis  are  arrayed  from  left  to  right  in  order  of  decreasing  "aggregate  influence" 
during  the  periods  of  dominance  by  the  group  named  under  the  column  on  the  extreme  left.  V- 
SAFT  computes  aggregate  influence  for  each  group  for  any  time  step  by  summing  the  individual 
influence  of  agents  activated  on  that  identity. 


Figure  7.  Domestic  political  hierarchy  levels  during  State  dominance.  Groups  are  shown  along  the  x- 
axis.  The  y-axis  is  the  number  of  model  time  steps  within  the  total  number  that  featured  the  State 
identity  group  as  dominant.  The  colored  bars  comprising  the  columns  report  the  proportion  of 
State-dominant  model  time  during  which  each  group  was  categorized  within  the  DPH  as  either  ally, 
opponent,  or  radical  opponent. 
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Figure  8.  Domestic  political  hierarchy  levels  during  Globalizing  dominance.  The  groups  are  shown 
along  the  x-axis.  The  y-axis  is  the  number  of  model  time  steps  within  the  total  number  that 
featured  the  "Globalizing"  (Western-oriented)  identity  group  as  dominant.  The  colored  bars 
comprising  the  columns  report  the  proportion  of  Globalizing-dominant  model  time  during  which 
each  group  was  categorized  within  the  DPH  as  either  ally,  opponent,  or  radical  opponent. 


As  noted  in  the  summary  forecast,  in  the  relatively  unlikely,  but  still  plausible,  circumstance  that 
business  itself  organizes  a  governing  coalition— relying  again  on  state  bureaucrats,  globalizers,  and 
criminal-corrupt  networks— prospects  for  intense  politics,  if  not  calm,  appear  to  improve 
considerably.  We  can  see  why  this  occurs  by  considering  the  display  in  Figure  9,  which  shows  that 
when  business  dominates,  the  largest  groups  in  Bangladesh  are  much  less  likely  to  adopt  a  radical 
oppositionist  posture  (the  blue  portion  of  columns  on  the  left  of  the  display)  toward  the 
government  than  when  state  bureaucrats,  globalizers,  or  criminal-corrupt  networks  (not  shown) 
dominate. 
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Figure  9.  Domestic  political  hierarchy  levels  during  Business  dominance.  The  groups  are  shown 
along  the  x-axis.  The  y-axis  is  the  number  of  model  time  steps  within  the  total  number  that 
featured  the  Business  identity  group  as  dominant.  The  colored  bars  comprising  the  columns  report 
the  proportion  of  Business-dominant  model  time  during  which  each  group  was  categorized  within 
the  DPH  as  either  ally,  opponent,  or  radical  opponent. 


Comparing  the  state  space  mapped  by  running  V-SAFT's  Bangladesh  model  for  the  year  beginning 
March  2013  to  mappings  produced  in  earlier  months  can  help  identify  general  trends.  For  example, 
in  Bangladesh,  Islamist-dominated  governments  appeared  prominently  enough  in  the  state  space 
to  justify  treating  that  outcome  as  plausible  in  each  of  the  six  previous  months'  forecasts  (meaning 
that  it  appears  in  at  least  3%  of  the  space  of  the  future).  For  the  year  beginning  in  March  2013, 
however,  the  likelihood  of  Islamist  governments  dropped  below  the  3%  threshold.  Accordingly, 
although  such  regimes  appear  in  the  display  of  plausible  governments  for  the  year  beginning 
February  2013  (on  the  left  in  Figure  10),  they  do  not  appear  in  the  display  on  the  right— for  the 
year  beginning  March  2013. 
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Figure  10.  Dominant  groups  in  Bangladesh.  The  x-axis  are  the  dominant  groups,  the  first  y-axis  is 
stability,  and  the  second  y-axis  is  the  percentage  of  the  space  in  which  the  forecast  holds.  For 
February  2013  (left)  x  group  appears  at  3%  and  in  March  2013  (right)  it  drops  below  that. 


On  the  other  hand,  by  noting  the  thin  gray  line  at  the  bottom  of  the  column  marked  "Muslim"  in 
the  "Political  Status  by  Group"  display  in  Figure  11,  we  can  see  that  V-SAFT  still  charted  a  Muslim- 
dominated  government  as  possible,  but  just  not  "plausible"  by  the  3%  rule. 
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Figure  11.  Political  status  by  group  in  Bangladesh.  This  graph  shows  the  likelihood  of  the  dominant 
part  of  government  coming  from  each  group  (x-axis). 


The  final  portion  of  the  summary  forecast  focuses  on  the  trend  toward  an  increasing  likelihood  of 
instability,  and  in  particular  of  severe  instability.  This  trend  is  most  easily  discerned  by  considering 
the  group  of  six  ''sequence  plots"  displayed  in  Figure  12a  -  12f,  beginning  with  the  forecasts  made 
for  the  October  2012-September  2013  year  and  ending  with  forecasts  made  for  the  March  2012- 
February  2014  year. 
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Figure  12a  &  12  b.  Six  "sequence  plots"— beginning  with  the  forecasts  for  October  2012  until 
February  2014.  Each  trajectory  (1,000)  is  traced  horizontally  by  a  single  line  to  represent  either 
calm  (green),  intense  politics  (blue),  instability  (yellow),  or  severe  instability  (red). 


Possible  Futures  -  Sorted  by  initial  conditions 
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Figure  12c  &  12d.  Six  "sequence  plots"— beginning  with  the  forecasts  for  October  2012  until 
February  2014.  Each  trajectory  (1,000)  is  traced  horizontally  by  a  single  line  to  represent  either 
calm  (green),  intense  politics  (blue),  instability  (yellow),  or  severe  instability  (red). 
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Figure  12e-  &  12f.  Six  "sequence  plots"— beginning  with  the  forecasts  for  October  2012  until 
February  2014.  Each  trajectory  (1,000)  is  traced  horizontally  by  a  single  line  to  represent  either 
calm  (green),  intense  politics  (blue),  instability  (yellow),  or  severe  instability  (red). 
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V-SAFT  produces  sequence  plots  such  as  those  in  Figure  12a-12f  by  stacking  all  1,000  trajectories. 
Each  week  in  each  trajectory,  traced  horizontally  by  a  single  line,  is  colored  to  represent  either  calm 
(green),  intense  politics  (blue),  instability  (yellow),  or  severe  instability  (red).  The  condition  present 
at  the  beginning  of  the  run  and,  within  that  group,  by  the  length  of  time  before  that  condition 
changes,  determines  the  order  of  the  stacking.  Different  horizontal  patterns  register  different 
sequences  of  stabilization  or  destabilization.  By  noting  the  changing  prominence  of  colors 
horizontally,  we  can  gain  a  quick  sense  of  time  trends  toward  stabilization  (becoming  more  green 
or  blue)  or  toward  destabilization  (becoming  more  yellow  or  red).  By  noting  the  changing 
prominence  of  colors  vertically,  we  can  gain  a  sense  of  how  much  of  the  state  space  of  the  future- 
taking  into  account  all  1,000  trajectories— is  relatively  stable  or  unstable  across  the  entire  square. 

By  comparing  the  balance  of  coloration  across  a  series  of  sequence  plots  for  the  same  country  we 
gain  a  sense  of  trends  in  V-SAFT  forecasts  for  that  particular  country.  These  can  include  increasing 
tendencies  toward,  for  example,  convergence  on  one  color  or  bifurcation— relative  disappearance 
of  yellow  or  blue  in  favor  of  either  calm  (green)  or  severe  instability  (red).  In  the  case  of  Bangladesh 
forecasts  for  the  year  beginning  with  October  2012  through  the  year  starting  in  March  2013,  we 
see  a  trend  toward  a  pronounced  shrinkage  in  the  areas  of  green  and  yellow  and  expansion  in  the 
areas  of  blue  and  especially  red,  suggesting  not  just  a  decreasing  probability  of  calm  but  an 
increasing  probability  of  either  intense  political  competition  or  severe  instability. 


5.  Conclusion 

Klein,  Moon,  &  Hoffman  (2006)  characterize  "sensemaking"  as  an  "umbrella  term  for  efforts  at 
building  intelligent  systems"  (p.  70).  They  imagined  that  such  systems  would  display  their 
intelligence  by  assembling  and  presenting  data  in  ways  that  evoke  understanding  of  the  data  as 
meaningful:  that  is  to  say  enlightening  or  satisfying  with  respect  to  the  ambitions,  questions,  or 
interests  of  users.  They  proceed  to  list  common  expectations  for  such  systems— expectations  that 
they  suggest  are  important,  but  difficult  to  fulfill.  Among  those  expectations  are  that  the  system 
will  automatically  fuse  massive  data  into  succinct  meanings,  process  meaning  in  contextually 
relative  ways,  enable  humans  to  achieve  insights,  automatically  infer  the  hypotheses  that  the 
human  is  considering,  enable  people  to  access  others'  intuitions,  and  present  information  in 
relevant  ways.  This  chapter  has  sought  to  offer  some  encouragement  with  respect  to  forecasting, 
analysis,  and  visualization  using  the  "social  radar"  metaphor  and  standard  questions  of  interest  to 
analysts  of  complex  societies.  Modeling  has  made  significant  progress  toward  satisfying  these 
expectations  in  the  eight  years  since  these  ambitions  were  listed,  although  much  work  remains  to 
be  done. 

Only  the  accumulation  of  a  large  data  set  of  forecasts  can  permit  the  kind  of  validation  tests  that 
alone  would  justify  commitment  of  resources  based  on  the  automatic  and  semi-automatic 
production  of  forecasts,  analyses,  and  visualizations  of  the  sort  now  available  with  V-SAFT.  The 
summary  forecasts,  which  use  narrative  techniques  to  weave  mnemonic  meaning  from 
standardized  but  still  dauntingly  complex  data  visualizations,  can  be  produced  automatically.  But 
this  will  require  considerable  work  based  on  studies  of  inter-coder  reliability,  dictionaries  of 
possible  combinations  of  reports  from  available  displays,  and  algorithms  to  produce  useful  and 
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sensible  prose  depictions  based  on  hierarchies  of  concern,  trends  along  key  dimensions,  and 
significant  anomalies.  It  is  difficult  to  imagine  that  this  distillation  process  could  be  completely 
automated,  since  one  key  aspect  of  forecasting  is  watching  for  events  that  could  happen  that  have 
never  happened  before.  Still,  given  the  progress  so  far,  one  could  quite  reasonably  expect  that  the 
kind  of  "intelligent  machine"  represented  by  V-SAFT  could  work  well  to  focus  the  attention  of 
human  analysts  on  the  kinds  of  problems  for  which  they  have,  and  will  always  have,  a  comparative 
advantage. 
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12  Training  for  sociocultural  forecasting:  Current  status  and 
science  and  technology  gaps1 

Winston  R.  Sieck,  Global  Cognition 


1.  Introduction 

Scientific  disciplines  across  the  board  have  struggled  with  the  challenge  of  anticipating  the  future. 
From  early  predictions  of  the  positions  of  planets  to  modern  meteorological  forecasting  systems, 
humans  have  witnessed  ongoing  improvements  in  the  ability  to  foresee  events.  Forecasting  within 
the  sociocultural  sphere  is  no  exception.  Although  many  scientists  are  highly  skeptical  about  the 
ultimate  prospects  of  forecasting  phenomena  in  the  area  of  human  social  cultural  behavior  with 
any  quantification  or  precision,  others  continue  to  plod  along,  making  small  gains  along  the  way. 
Research  programs  that  make  any  attempt  to  improve  forecasting  are  highly  susceptible  to  being 
labeled  as  failures,  at  least  in  part  because  they  are  plagued  from  the  outset  by  unrealistic 
expectations  of  what  they  can  accomplish  in  the  short  term.  There  is,  however,  a  large  difference 
between  only  being  able  to  make  a  measured  amount  of  real  progress  in  the  short  term,  and  being 
unable  to  achieve  anything  at  all.  This  chapter  describes  the  current  state  of  the  science  and 
technology  related  to  training  in  sociocultural  forecasting,  and  identifies  gaps  that  research  should 
address  to  develop  capabilities  for  application  in  operational  settings. 

The  following  incident  illustrates  the  intuitive  use  of  native-level  cultural  knowledge  to  anticipate  a 
political  leader's  actions.  It  provides  several  interesting  points  for  discussion,  not  the  least  of  which 
concerns  how  training  in  sociocultural  forecasting  techniques  could  lead  to  the  development  of  a 
similar  depth  of  understanding  among  non-native  analysts  and  operators,  at  least  about  relatively 
narrow  forecasting  problems  like  this  one  (Sieck,  McHugh,  Klein,  Wei,  &  Klinger,  2004,  p.  19): 

The  order  came  down  to  the  head  of  the  Bosnian  analysis  team  on  Sunday  morning.  The 
General  needed  the  team  to  identify  the  propaganda  themes  that  the  Croats,  Bosniacs,  and 
Serbs  would  use  by  Monday.  The  team  came  in  on  a  Sunday  to  put  together  a  briefing. 

The  team  was  multinational,  with  an  American,  a  Dutch,  a  Turk,  a  Greek,  and  a  German  officer 
all  working  together.  The  American  was  the  team  leader.  They  had  been  studying  the  issue 
and  background  data  for  about  nine  weeks.  The  team  had  two  separate  interpretations  and 
analyses  of  what  was  happening  and  what  was  driving  events.  The  critical  difference  had  to 
do  with  the  Serbs,  and  the  possibility  that  Milosevich  would  switch  from  Stalinist  communism 
to  pursue  a  religious  angle  in  the  upcoming  months. 


1  Approved  for  Public  Release;  Distribution  Unlimited.  Case  Number  14-2487 
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Most  of  the  team  members  looked  at  the  situation  as  purely  a  political  process,  Milosevich 
was  promising  the  people  who  were  his  natural  constituency  a  better  day.  They  predicted 
that  he  would  continue  to  promise  a  greater  Serbia,  with  the  Serbs  dominating  the  political 
processes  and  the  economic  base  of  whatever  was  left  of  Yugoslavia.  At  dinner,  the  Greek  and 
the  Turkish  officers  got  frustrated  with  the  conversations  of  the  others  and  finally  said,  "y°u 
don't  understand."  And  the  Turkish  officer  said,  "look  the  problem  is  you're  thinking  secular, 
and  they're  not."  At  this  point,  the  reaction  of  the  others  was  silence.  It  was  one  of  those 
conversation  killers.  The  Greek  and  Turk  argued  that  the  political  message  was  no  longer 
powerful  enough  to  rally  the  Serbs,  because  there  was  no  passion  in  it.  They  said  that  the  only 
thing  passionate  in  this  was  the  religious  separation.  The  Dutchman  said  that  it  might  work, 
though  he  didn't  believe  it.  He  just  didn't  believe  that  a  Marxist  could  get  away  with  taking 
that  line,  or  that  he  would  have  any  credibility  if  he  tried. 

After  dinner  the  leader  was  approached  by  the  Greek  and  Turk  again.  The  team  leader 
listened  to  them  because  it  struck  him  as  very  unusual  that  people  from  traditional  enemy 
countries  came  to  the  same  conclusion  and  worked  together  to  get  the  word  to  him.  Also,  he 
recognized  that  they  had  a  very  good  feel  for  the  culture,  much  better  than  the  rest  of  the 
team.  The  rest  had  the  distant  political-analytical  perspective,  but  they  had  a  much  closer, 
more  personal,  visceral  perspective.  The  leader  included  the  religious  angle  for  the  Serbs.  In 
the  end,  Milosevich  did  take  the  religious  angle.  He  made  his  first  connection  to  religion  in 
this  conflict  in  May  of  1992,  several  weeks  after  their  prediction. 

This  case  of  sociocultural  forecasting  by  a  multinational  intelligence  team  illustrates  several  aspects 
relevant  to  sociocultural  forecasting: 

•  Like  most  forecast  problems,  it  involves  the  prediction  of  some  change  from  the  status  quo. 

•  The  prediction  made  is  categorical  rather  than  probabilistic;  the  possibility  of  a  religious 

theme  would  either  be  included  or  not,  with  no  probability  attached. 

•  Several  team  members  seem  extremely  confident  in  their  predictions,  perhaps  even 
overconfident. 

•  The  Greek,  Turkish,  and  Dutch  members  of  the  team  all  had  substantial  sociocultural 

knowledge,  but  that  knowledge  was  of  different  kinds.  The  Greek  and  Turkish  members 
were  apparently  better  able  to  adopt  the  "native  perspective"  on  the  situation, 
appreciating  what  the  Serbian  population  would  and  would  not  support. 

This  last  point— about  the  potential  value  of  understanding  how  people  from  a  culture  of  interest 
think  and  make  decisions  before  attempting  to  forecast  their  future  behavior— raises  especially 
interesting  questions.  This  chapter  addresses  how  sociocultural  forecasters  could  be  trained  to 
make  accurate  predictions  of  culturally  different  others  without  the  "visceral"  understanding  of  the 
culture  that  only  natives,  and  perhaps  a  few  long-term  expatriates,  would  share.  It  describes 
selected  current  science  and  technology  in  this  area,  as  well  as  recommended  future  efforts  to 
address  this  question. 
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The  next  section  provides  an  overview  of  literature  on  probabilistic  forecasting,  with  an  emphasis 
on  training  analysts  how  to  perform  probabilistic  forecasting.  It  then  outlines  special  problems 
associated  with  forecasting  in  the  sociocultural  domain,  along  with  their  training  implications. 
These  include  the  nature  and  range  of  problems  being  forecast,  from  short-term  political  actions 
and  events  to  long-term  changes  in  population  cultural  values,  beliefs,  and  behaviors.  This  section 
also  discusses  difficulties  in  representing  emergent  processes  for  forecasting,  such  as  those  that 
underlie  sociocultural  systems,  and  describes  cultural  resilience  to  aid  in  explaining  why  cultural 
change  frequently  does  not  happen  or  often  has  only  temporary  effects. 

The  third  section  presents  an  overview  of  literature  and  capabilities  that  support  sociocultural 
forecasting  and  training.  This  includes  work  on  concepts  for  training  in  cultural  sensemaking  and 
their  relation  to  sociocultural  forecasting,  as  well  as  training  in  the  use  of  cognitive-cultural  models 
for  forecasting  sociocultural  behavior.  In  addition,  it  covers  training  implications  of  the  Defense 
Advanced  Research  Projects  Agency  (DARPA)  and  HSCB-funded  Worldwide  Integrated  Crisis  Early 
Warning  System  (W-ICEWS)  program,  as  well  as  forecasting  training  efforts  within  the  Intelligence 
Advanced  Research  Projects  Activity's  (IARPA)  Aggregative  Contingent  Estimation  (ACE)  program. 

The  fourth  section  summarizes  science  and  technology  gaps  in  this  area  that  research  has  yet  to 
address.  The  chapter  concludes  with  a  description  of  recommended  next  steps  to  move  training 
capabilities  toward  operational  usage. 


2.  Training  Probabilistic  Forecasting 

Several  research  efforts  have  attempted  to  determine  how  best  to  train  people  to  make  better 
probabilistic  forecasts.  In  general,  they  seek  to  train  analysts  in  the  process  by  which  people 
generate  forecasts,  and  to  determine  whether  specific  types  of  feedback  lead  to  improved 
probabilistic  forecasts.  Such  procedures  could  then  be  incorporated  into  forecasting  tools  and 
implemented  within  formalized  training  and  education  programs  for  specialists. 

Probabilistic  forecasting  has  the  advantage  of  enabling  forecasters  to  express  uncertainty  in  their 
opinions  using  precise  language  that  permits  application  of  various  quantitative  metrics  to  assess 
the  accuracy  of  sets  of  forecasts  (Lehner,  Michelson,  &  Adelman,  2010;  Tetlock,  2005).  These 
accuracy  measures  allow  formal  evaluation  of  forecasting  systems,  and  also  constitute  a  primary 
basis  for  generating  different  kinds  of  feedback  to  use  in  training  forecasters. 

Perhaps  the  most  common  measure  of  the  accuracy  of  probabilistic  forecasts  is  the  mean 
probability  score  or  "Brier  score"  (Brier,  1950;  Yates,  1994).  To  obtain  the  Brier  score  over  a  set  of 
forecasts,  the  probability  score  for  each  forecast  is  first  calculated  as  (f-d )2,  where /is  the  reported 
likelihood  that  the  target  event  will  occur  (e.g.,  the  forecaster's  stated  probability  that  Iran  and  the 
United  States  will  commence  official  nuclear  program  talks  before  1  October  2013),  and  d  is  the 
actual  outcome  (coded  as  0  if  the  target  event  does  not  occur  and  1  if  the  target  event  does  occur). 
Probability  scores  are  calculated  for  each  of  many  forecasts,  and  these  are  averaged  to  obtain  the 
Brier  score.  As  the  equation  shows,  the  Brier  score  is  a  measure  of  error  in  probability  forecasts,  so 
the  lower  the  Brier  score,  the  better  the  forecasts.  Researchers  have  identified  various  means  for 
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decomposing  the  Brier  score  into  more  fine-grained  components  that  offer  greater  potential  for 
detecting  certain  kinds  of  biases,  and  for  formulating  training  applications  based  on  feedback  from 
accuracy  measures  (e.g.,  Yates,  1982). 

The  two  most  commonly  discussed  subcomponents  of  probability  forecast  accuracy  are  calibration 
and  discrimination  (see  Figure  1).  Calibration  measures  the  extent  to  which  a  forecaster  states 
probabilities  that  match  various  base  rates,  or  the  percentages  of  times  that  the  target  event 
actually  occurs.  Often,  percentages  of  event  occurrence  are  plotted  for  each  probability  judgment 
to  produce  a  calibration  curve  (Lehner,  Michelson,  &  Adelman,  2010).  A  forecaster  might  exhibit 
poor  calibration  in  several  ways,  primarily  over-prediction  and  overconfidence.  Over-prediction 
means  that  the  average  of  the  forecaster's  reported  probabilities  exceeds  the  actual  proportion  of 
times  that  the  target  events  occurred.  For  example,  suppose  an  analyst  reports  probability 
forecasts  for  50  events  such  as  the  U.S.-lran  event  given  above.  If  the  average  of  the  analyst's 
probabilities  is  70%,  and  yet  only  25%  of  the  events  actually  occur,  then  the  analysts'  forecasts  are 
exhibiting  over-prediction.  Overconfidence  typically  refers  to  situations  in  which  the  analysts' 
forecasts  are  too  extreme.  To  illustrate  overconfidence,  imagine  a  two-step  procedure  in  which  an 
analyst  first  states  a  categorical  prediction  as  to  whether  or  not  each  event  will  occur.  The  analyst 
then  reports  the  probability  that  his  or  her  categorical  prediction  is  correct.  If,  over  a  set  of  50 
problems,  the  forecaster  is  correct  in  58%  of  his  or  her  categorical  predictions,  yet  reports  an 
average  probability  correct  of  90%,  then  extreme  overconfidence  is  evident.  The  two-step 
procedure  aids  in  illustrating  overconfidence,  although  the  measure  and  the  elicitation  method  are 
independent.  Overconfidence  can  be  computed  regardless  of  how  the  probability  forecasts  are 
reported. 
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Figure  1.  Subcomponents  of  probability  accuracy. 
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The  second  general  subcomponent  of  accuracy,  discrimination,  refers  to  a  forecaster's  ability  to 
distinguish  between  situations  when  the  target  event  will  occur  and  when  it  will  not  occur.  A 
simple  measure  of  discrimination  is  the  "slope"  statistic  proposed  by  Yates  (1982).  Slope  is  the 
mean  of  the  probability  forecasts  for  which  the  events  occurred,  minus  the  average  of  probability 
forecasts  for  which  the  events  did  not  occur.  The  higher  the  slope,  the  better  a  person's  forecasts 
distinguish  between  events  that  occur  from  those  that  do  not. 

It  is  important  to  recognize  that  a  forecaster  could  do  well  on  one  of  these  aspects  of  probability 
accuracy  but  poorly  on  the  other.  That  is,  calibration  and  discrimination  are  distinct  components  of 
overall  forecast  accuracy.  This  has  led  researchers  to  examine  whether  the  calibration  and 
discrimination  aspects  of  forecasting  accuracy  require  related  or  distinct  cognitive  skills,  for 
example,  by  attempting  to  determine  whether  training  procedures  that  affect  one  of  the  measures 
also  produce  effects  on  the  second  measure  (Stone  &  Opel,  2000). 

3.  Training  to  Improve  Calibration  and  Discrimination 

Most  training  in  probability  forecasting  has  emphasized  improving  calibration,  presumably  because 
it  is  generally  viewed  as  simpler  to  achieve.  Perhaps  the  most  intensive  training  intervention  for 
probability  judgment  using  calibration  feedback  was  conducted  by  Lichtenstein  and  Fischhoff 
(1980).  In  their  study,  trainees  completed  elevent  training  sessions  involving  200  judgment 
problems.  At  the  completion  of  each  training  session,  the  trainees  received  feedback  on  their 
probability  accuracy  that  included  graphs  to  show  their  calibration,  as  well  as  detailed  performance 
statistics  such  as  the  Brier  score  and  numerical  measures  of  calibration.  The  researchers  explained 
and  discussed  the  feedback  with  the  trainees  for  five  to  ten  minutes.  Following  this  rather  intensive 
intervention,  trainees  showed  a  clear  improvement  in  calibration. 

Researchers  have  delineated  several  types  of  feedback  potentially  relevant  to  forecast  training 
(Benson  &  Onkal,  1992).  Giving  trainees  the  results  of  their  probability  accuracy  measures,  as  in  the 
example  above,  represents  just  one  kind  of  feedback,  which  is  sometimes  referred  to  as 
"performance  feedback"  (Benson  &  Onkal,  1992).  Several  researchers  have  suggested  that 
performance  feedback  is  a  particularly  effective  method  for  improving  calibration  under  suitable 
conditions  (e.g.,  by  reducing  overconfidence).  However,  performance  feedback  has  not  proven 
particularly  successful  at  increasing  discrimination,  since  it  contains  no  substantive  information 
that  would  help  trainees  to  determine  whether  or  not  a  particular  event  will  occur. 

In  order  to  disentangle  the  issues,  researchers  have  also  delineated  another  type  of  feedback 
potentially  relevant  to  forecast  training:  environmental  feedback  (Benson  &  Onkal,  1992).  Whereas 
performance  feedback  gives  trainees  information  about  the  accuracy  of  their  forecasts,  including 
components  such  as  measures  of  overconfidence,  over-prediction,  or  discrimination, 
environmental  feedback  provides  information  about  the  event  to  be  predicted.  For  our  purposes, 
environmental  feedback  refers  to  any  domain-specific  information  relevant  to  the  forecasting  task, 
such  as  associations  between  cues  and  outcomes.  Various  forms  of  cultural  information  relevant  to 
a  leader's  decisions,  for  example,  would  be  especially  relevant  to  predictions  in  the  sociocultural 
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realm.  In  contrast  to  performance  feedback,  environmental  feedback  appears  to  improve 
discrimination  but  not  calibration. 

In  one  study  designed  to  test  the  relative  benefits  of  distinct  kinds  of  feedback,  participants  were 
asked  to  provide  forecasts  and  given  performance  feedback.  One  group  of  participants  received 
feedback  on  their  calibration,  while  another  group  received  feedback  on  their  discrimination 
scores.  These  both  reflect  performance  feedback,  as  they  relay  information  as  to  how  well  the 
trainee  forecast  in  the  problem  domain.  As  in  earlier  work,  performance  feedback  regarding 
calibration  improved  participants'  calibration  scores  but  had  no  impact  on  discrimination  (Benson 
&  Onkal,  1992). 

Subsequently,  Stone  and  Opel  (2000)  explicitly  contrasted  the  impact  of  environmental  and 
performance  feedback  on  the  accuracy  of  probability  judgments.  They  randomly  assigned 
participants  to  performance  feedback,  environmental  feedback,  or  no  feedback  (control) 
conditions.  Their  study  procedure  included  three  parts:  pre-training  probability  judgments, 
training,  and  post-training  probability  judgments.  The  performance  feedback  group  received 
individualized  feedback  regarding  their  calibration  performance.  The  environmental  feedback 
group  received  information  in  the  form  of  a  30-minute  lecture  on  a  historical  subject  designed  to 
increase  their  substantive  knowledge  about  the  topic  of  their  predictions.  All  participants  received 
a  handout  that  described  five  to  ten  important  characteristics  relevant  to  the  topic  at  hand.  They 
then  practiced  using  the  new  knowledge  to  re-analyze  20  of  their  probability  judgment  problems 
from  the  pre-training  session.  The  control  (no  feedback)  group  completed  an  unrelated  task  during 
the  training  period. 

Stone  and  Opel  confirmed  that  performance  feedback  led  to  improvements  in  calibration,  but  not 
discrimination,  and  that  environmental  feedback  led  to  improvements  in  discrimination,  but  not 
calibration.  Importantly,  the  researchers  also  found  that  substantive  training  in  the  form  of 
providing  additional  environmental  cues  and  information  can  actually  degrade  calibration.  In 
particular,  after  participants  received  environmental  feedback,  their  overconfidence  increased 
(Stone  &  Opel,  2000).  The  latter  finding  suggests  that  merely  increasing  people's  knowledge  about 
a  particular  area  or  domain,  for  instance  by  encouraging  analysts  to  digest  volumes  of  information 
about  the  culture  of  a  region  of  interest,  can  lead  to  an  increase  in  forecast  confidence  not 
matched  by  commensurate  increases  in  accuracy. 

Within  the  domain  of  sociocultural  forecasting,  this  implies  that  knowledge  of  the  culture  or  area  is 
insufficient  to  accurately  anticipate  future  events.  Instead,  training  programs  must  also  incorporate 
forecast  performance  feedback  using  measures  of  calibration.  This  represents  an  important  issue 
from  a  practical  standpoint,  as  it  indicates  that  training  methods  that  attempt  to  influence  one 
component  of  accuracy  (calibration  or  discrimination)  may  not  have  any  impact  on  the  other 
component.  Table  1  summarizes  these  relationships. 
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Table  1.  Types  of  Probability  Feedback  and  Their  Effects  on  Performance 


Type  of 

Feedback 

Description 

Effects  on  Accuracy 

Performance 

Information  about  components  of 
forecast  accuracy  (calibration, 
discrimination) 

•  Improves  calibration 

•  No  effect  on  discrimination 

Environmental 

Information  about  the  event  to  be 

predicted,  such  as  cultural  information 

•  Can  improve  discrimination 
if  information  is  diagnostic 

•  Can  harm  calibration  by 
increasing  overconfidence 

Stone  and  Opel  concluded  that  calibration  and  discrimination  reflect  distinct  cognitive  skills,  which 
they  called  calibration  expertise  and  substantive  expertise.  Substantive  expertise  refers  to  domain- 
specific  knowledge  in  a  certain  area,  such  as  sociocultural  expertise  in  a  given  region.  Calibration 
expertise  reflects  the  ability  to  render  probability  forecasts  that  exhibit  neither  over-  or  under¬ 
prediction  nor  under  or  overconfidence.  Furthermore,  distinct  types  of  feedback  and  training  are 
required  to  improve  these  different  cognitive  skills  and  thus  promote  overall  expertise  in 
probabilistic  sociocultural  forecasting. 


4.  Forecasting  in  the  Sociocultural  Domain 

This  section  describes  issues  associated  with  forecasting  in  the  sociocultural  domain,  as  well  as 
their  training  implications.  For  illustration,  this  section  employs  an  example  regarding  crowd 
behavior  in  the  Middle  East  and  the  Arab  Spring. 

One  issue  concerns  the  scope  of  problems  being  forecast,  which  can  range  from  immediate 
reactions  to  medium-term  political  actions  and  events  to  long-term  change  in  population  cultural 
values,  beliefs,  and  behaviors.  First,  analysts  might  use  sociocultural  knowledge  and  tools  to  predict 
local,  immediate,  transitory  outcomes,  such  as  newsworthy  events  of  the  day.  For  instance,  Sieck, 
Simpkins,  and  Rasmussen  (2013)  investigated  crowd  member  responses  to  security  force  actions  in 
the  Middle  East.  The  cultural,  behavioral,  and  situational  factors  they  examined  should  aid  in 
anticipating  whether  an  Arab  crowd  would  turn  violent  or  remain  peaceful  in  a  time  span  of  a  few 
moments  to  a  few  hours.  Training  based  on  those  models  would  enable  U.S.  personnel  to  make 
sense  of,  anticipate,  and  hence  more  effectively  manage  crowds  in  places  such  as  Iraq  (Sieck, 
Smith,  Grome,  &  Rababy,  2010).  Such  training  would  also  likely  have  aided  an  analyst  to  track 
events  as  they  unfolded  during  the  Arab  Spring  to  forecast  the  likelihood  of  violent  eruptions  on 
the  same  time  scale. 
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At  a  second  level,  sociocultural  sensemaking  might  be  used  to  predict  outcomes  in  the  middle 
term,  such  as  changes  in  political  leadership.  An  example  at  this  level  would  be  an  analyst  forecast 
of  the  likelihood  that  Hosni  Mubarak  would  not  remain  as  Egypt's  leader  as  a  result  of  the  Arab 
Spring  protests. 

Finally,  a  third  level  concerns  distant,  long-term  outcomes,  such  as  changes  in  deep-rooted  cultural 
values.  A  forecast  problem  at  this  level  might  address  the  likelihood  that  the  Arab  Spring  and 
changes  in  political  institutions  within  Egypt  would  reduce  the  cultural  value  of  "power-distance" 
over  the  next  decade.  Power-distance  measures  the  degree  to  which  members  of  the  culture 
accept  and  expect  unequal  distribution  of  power  in  the  society  (Hofstede,  2001).  People  in  high 
power-distance  cultures  tend  to  be  comfortable  with  the  idea  that  their  leaders  (at  the  national, 
organizational,  or  family  level)  hold  the  lion's  share  of  power.  Middle  Eastern  countries  tend  to 
rank  very  high  on  Hofstede's  power-distance  index  (Hofstede,  2001).  For  instance,  if  the  cultural 
value  placed  on  power-distance  remains  constant  within  Egypt,  new  governments  and  leadership 
would  probably  assume  just  as  much  control  as  their  predecessors,  since  this  is  expected  at  a  deep 
level  by  their  constituents. 

To  date,  most  research  and  application  of  sociocultural  forecasting  has  apparently  focused 
primarily  on  the  middle  level.  However,  outcomes  at  this  level  may  well  not  mean  what  we  think 
they  mean  in  terms  of  longer  term,  strategic-level  implications.  Sociocultural  forecasting  and 
persistent  sensing  of  fundamental  cultural  values  that  represent  long-term  changes  to  societies 
constitute  an  important  area  for  further  applied  research  and  development. 

Consideration  of  long-term,  fundamental  cultural  values  leads  naturally  to  a  second  issue  in 
sociocultural  forecasting:  the  need  to  understand  why  cultural  change  frequently  does  not  happen 
or  often  has  only  temporary  effects.  An  important  factor  associated  with  a  culture's  resistance  to 
change  is  the  looseness  or  tightness  with  which  its  members  hold  on  to  cultural  norms:  the 
standards  by  which  members  of  the  culture  live.  These  norms  represent  the  shared  expectations 
and  rules  that  guide  the  behavior  of  people  within  social  groups,  and  as  such  are  learned  from  and 
reinforced  by  parents,  friends,  teachers,  and  others  as  children  grow  up  in  a  society. 

Gelfand  and  a  large  international  team  investigated  cross-cultural  differences  in  the  extent  to 
which  cultural  norms  matter  to  the  members  of  the  society  (Gelfand  et  al.,  2011).  Some  societies 
exhibit  "cultural  tightness,"  insisting  on  strong  conformity  to  their  cultural  norms  in  all  areas. 
Other,  "culturally  loose"  societies  tolerate  far  more  deviance  from  the  norms.  Gelfand  and 
colleagues  theorized  that  tightness  and  looseness  are  reflected  at  different  levels  within  a  culture 
that  mutually  support  one  another.  Specifically,  Gelfand  et  al.  (2011)  described  evidence  related  to 
four  levels: 

1.  Ecological  and  Historical  Threats.  Hostile  neighbors,  disease,  and  dense  populations 
increase  the  need  for  coordinated  and  disciplined  action  from  the  population.  Factors  such 
as  these  tighten  cultural  norms.  As  the  threats  diminish,  adherence  to  norms  becomes 
looser. 
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2.  Sociopolitical  Institutions.  Culturally  tight  nations  tend  to  have  more  autocratic 
governments,  restricted  media,  stronger  suppression  of  dissent,  and  more  severe 
punishments  for  crime. 

3.  Everyday  Social  Situations.  All  kinds  of  interactions  with  fellow  members  of  the  culture  are 
more  formal  in  nations  with  tight  cultural  norms.  These  include  situations  at  home,  the 
workplace,  school,  places  of  worship,  parks,  and  others.  Loose  cultures  provide  more  room 
for  individual  discretion  in  such  situations;  a  wider  range  of  behavior  qualifies  as 
"appropriate/' 

4.  Psychological  Adaptations.  People's  minds  become  attuned  to  the  different  requirements 
of  living  in  places  with  tight  or  loose  cultural  norms.  Individual  psychology  then  further 
supports  the  level  of  cultural  tightness  or  looseness.  People  living  in  tight  cultures  become 
more  focused  on  avoiding  mistakes,  are  more  cautious  in  their  own  behavior,  and  more 
closely  monitor  themselves  and  others  for  norm  violations. 

A  sociocultural  forecaster  might  take  many  cultural  values  and  factors  into  account  when 
generating  a  prediction.  Cultural  tightness  or  looseness  of  norms  represents  one  of  a  small  set  of 
potential  factors  that  specifically  address  tolerance  for  deviance  and  hence  willingness  to  change  at 
the  cultural  level.  It  is  therefore  a  potentially  useful  factor  to  consider  across  many  forecast 
problems.  In  addition,  the  Gelfand  team's  analysis  of  the  issue  in  terms  of  the  four  mutually 
supporting  levels  provides  an  excellent  illustration  of  the  general  factors  involved  in  maintaining 
any  cultural  system  in  its  current  state. 


5.  Training  in  Sociocultural  Forecasting 

Several  programs  have  addressed  topics  related  to  training  in  sociocultural  sensemaking  and 
forecasting,  including  the  Office  of  the  Secretary  of  Defense's  Human  Social  Culture  Behavior 
(HSCB)  Modeling  Program,  lARPA's  ACE  program,  and  DARPA's  W-ICEWS  program.  This  section 
discusses  selected  research  and  development  topics  from  each  of  these  programs. 

5.1.  HSCB  modeling  program 

Cognitive-Cultural  Modeling 

The  HSCB  program  has  sponsored  the  use  of  cognitive-cultural  models  that  can  aid  in  forecasting 
sociocultural  behavior.  Cognitive-cultural  models  are  graphical  representations  of  a  culture's 
shared  values,  conceptions,  and  causal  beliefs  that  influence  decisions  by  members  of  interest  in 
that  culture  (Sieck,  2010;  Sieck,  Rasmussen,  &  Smart,  2010).  These  models  help  cultural  outsiders 
to  assume  the  viewpoint  of  cultural  insiders.  Cognitive-cultural  models  also  aid  in  identifying 
cultural  elements  that  should  receive  high  priority  in  training,  and  in  anticipating  the  behavior  of 
the  members  of  the  culture.  In  their  most  advanced  form,  cognitive-cultural  models  also  represent 
quantitative  information  about  the  prevalence  of  the  ideas  included. 

Researchers  developed  the  models  via  Cognitive-Cultural  Analysis  (CCA).  This  process  was  originally 
termed  "Cultural  Network  Analysis,"  but  the  name  was  changed  to  highlight  the  cognitive  emphasis 
of  the  models,  i.e.,  that  the  models  seek  to  reveal  how  members  of  the  culture  think  and  make 
decisions.  The  CCA  approach  adopts  the  theory  of  "cultural  epidemiology,"  which  implies  that  ideas 
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can  be  studied  using  some  of  the  same  techniques  that  epidemiologists  use  to  study  diseases 
(Berger  &  Heath,  2005;  Sperber,  1985). 

CCA  ensures  that  the  cognitive  aspects  of  a  culture  can  be  incorporated  into  practical  use,  such  as 
forecasting  and  training  applications,  by  identifying  key  decisions  or  judgments  of  interest  as  a  first 
step.  Once  the  key  decisions  have  been  determined,  cultural  analysts  construct  models  which 
include  the  cultural  ideas  that  directly  influence  those  decisions. 

CCA  encompasses  several  techniques  needed  to  build  cognitive-cultural  model  diagrams.  The 
primary  representation  format  for  CCA  is  the  influence  diagram,  used  very  successfully  for  some 
time  to  map  knowledge  in  decision  analysis  (Howard,  1989).  Research  has  developed  techniques  to 
(a)  elicit  concepts,  causal  beliefs,  and  values  from  people  in  interviews  or  survey  instruments,  (b) 
extract  the  ideas  from  interview  transcripts  or  other  text  material  (e.g.,  text  harvested  from  the 
World-Wide  Web),  (c)  analyze  the  degree  of  commonality  in  ideas  between  cultural  groups,  (d) 
align  and  assemble  consensus  groups  of  ideas  into  maps,  and  (e)  relate  them  to  demographic 
variables.  Although  CCA  seeks  to  represent  the  idea  networks  in  a  common,  scientific  format,  it 
nonetheless  maintains  the  content  of  cultural  knowledge  as  expressed  by  members  of  the  cultural 
group  (Sieck,  2010).  This  is  exactly  the  information  that  analysts  need  to  acquire  the  visceral 
understanding  described  in  the  illustrative  case  at  the  beginning  of  this  chapter. 

The  resulting  cognitive-cultural  models  include  estimates  of  the  prevalence  of  such  cultural  ideas. 
Capturing  the  proportions  of  people  who  actually  maintain  the  relevant  beliefs  provides  a  full 
description  of  the  current  cognitive  state  of  a  culture.  With  respect  to  prediction,  prevalence 
information  yields  relevant  status  quo  base-rates  to  which  forecasters  can  anchor  and  then  adjust 
to  ensure  forecast  realism.  In  this  way,  cognitive-cultural  models  that  include  quantitative 
estimates  of  idea  prevalence  give  forecasters  environmental  feedback  in  a  form  that  has  the 
potential  to  improve  both  aspects  of  probability  accuracy:  discrimination  and  calibration. 

CCA  is  ideally  suited  to  simulations  using  Bayesian  modeling,  although  the  application  is  somewhat 
different  from  Bayesian  modeling  applications  that  rely  on  expert  inputs.  In  standard  expert 
applications  of  Bayesian  belief  networks  (BNs),  researchers  elicit  structural  and  probability  inputs 
from  experts  to  determine  the  representation  of  a  physical  system,  with  the  aim  of  accurately  and 
quantitatively  predicting  key  physical  outcomes  based  on  the  experts'  understanding  of  influences 
within  the  system.  By  contrast,  in  a  cognitive-cultural  modeling  application  the  aim  is  to  capture 
cultural  knowledge  in  order  to  accurately  anticipate  the  perceptions  and  decisions  of  members  of 
cultural  groups.  The  key  principles  governing  application  of  BNs  to  cultural  modeling  are  that  BNs 
focus  on  the  population,  rather  than  on  the  individual  psychological  level  of  analysis;  nodes 
represent  concepts  held  in  common  by  some  percentage  of  the  population;  edges  represent  causal 
beliefs,  also  distributed  across  members  of  the  population;  and  probabilities  denote  the  prevalence 
of  ideas  in  the  population,  not  the  strength  of  belief. 

Projects  sponsored  by  the  HSCB  Program  have  used  CCA  to  develop  several  quantitative  cognitive- 
cultural  models  of  Afghan  decision  making  based  on  interviews  and  a  survey  study  of  400  Afghans 
from  several  different  provinces  (Sieck,  Javidan,  Osland,  &  Rasmussen,  2012).  CCA  and  one  of  the 
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Afghan  cognitive-cultural  models  were  also  used  in  the  development  of  a  prototype  tool  for 
simulating  cultural  behavior,  known  as  the  Cultural  Belief  Network  Simulation  Tool  (CuIBN)  (Sieck, 
Simpkins,  &  Rasmussen,  2011).  CuIBN  provides  a  user  interface  that  enables  the  creation, 
visualization,  and  manipulation  of  input  values  in  a  cognitive-cultural  model.  The  interface  was 
combined  with  a  standard  BN  simulation  engine  that  could  generate  forecasts  contingent  on 
specific  changes  within  the  model  (Sieck,  Simpkins,  &  Rasmussen,  2011).  The  system  allowed 
forecasters  to  interact  with  the  model,  for  instance  by  entering  hypothesized  changes  in  certain 
elements  and  then  visualizing  the  predicted  consequences  in  terms  of  quantitative  adjustments  to 
cultural  prevalence  values.  Some  of  the  primary  anticipated  benefits  of  CuIBN  in  supporting  the 
training  of  forecasters  include  that  it  provides  a  coherent  approach  to  constraining  and  making 
sense  of  a  vast  array  of  concepts,  causal  beliefs,  and  values,  as  they  are  relevant  to  particular 
decisions  and  behaviors  of  interest  to  the  forecaster. 

One  of  the  challenges  for  novice  sociocultural  forecasters  is  managing  the  overload  of  information 
potentially  relevant  to  a  forecast.  CuIBN's  structure  may  help  reduce  the  amount  and  increase  the 
relevance  of  information  that  analysts  must  take  into  consideration,  which  should  support  both 
forecast  calibration  and  discrimination.  In  addition,  experienced  analysts  can  usefully  share 
cognitive-cultural  models  to  provide  environmental  feedback  that  can  aid  novice  analysts  in  gaining 
expertise  on  specific  topics  relevant  to  their  area  of  study.  These  possibilities  should  be  tested  in 
applied  research  settings. 

Training  in  Culturai  Sensemaking 

Training  in  cultural  sensemaking  provides  learners  with  cultural  knowledge  relevant  to  explaining 
culture-specific  behavior,  as  well  as  metacognitive  strategies  for  coping  with  unexpected  behaviors 
and  consequently  acquiring  new  knowledge  (Rasmussen  &  Sieck,  2012;  Rasmussen,  Sieck,  & 
Osland,  2010).  Such  training  may  use  cognitive-cultural  models  to  provide  direct  and  specific  input 
to  support  novice  thinking.  Rasmussen  et  al.  (2010)  outlined  a  theoretical  framework  for  cultural 
sensemaking  that  connects  metacognitive  skills  to  region-specific  knowledge.  They  also  described  a 
novel  approach  to  instructional  analysis  and  design,  specifically  developed  to  identify  learning 
objectives  and  content  for  training  in  cultural  sensemaking.  Demonstration  of  the  method  led  to 
the  development  of  a  training  booklet  and  materials  that  some  military-cultural  instructors  have 
incorporated  into  their  courses  (Rasmussen  &  Sieck,  2010). 

Within  this  theoretical  framework,  cultural  sensemaking  refers  to  the  processes  by  which  people 
make  sense  of  and  explain  culturally  different  behaviors  (Osland  &  Bird,  2000).  In  such  cases,  it  is 
natural  that  peoples'  initial  perspectives  are  driven  by  expectations  based  on  normal  behavior 
learned  within  their  own  culture  (Archer,  1986).  An  initial  challenge  within  a  cultural  sensemaking 
situation  or  training  simulation  is  recognizing  when  the  models  one  would  normally  use  no  longer 
apply.  Next,  individuals  seek  the  information  they  need  in  order  to  develop  culture-appropriate 
understanding  of  the  current  situation.  This  provides  a  basis  for  projecting  likely  subsequent 
actions  and,  in  the  present  context,  for  developing  informed  probabilistic  forecasts. 
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An  important  metacognitive  skill  to  support  cultural  sensemaking  is  the  ability  to  build  the 
knowledge  required  to  explain  and  predict  behavior.  Further,  this  overall  skill  is  embedded  within  a 
framework  of  related  metacognitive  skills  that  allow  individuals  to  obtain,  apply,  test,  and  refine 
their  cultural  knowledge.  These  metacognitive  skills  are  culture  general  in  the  sense  they  support 
attainment  of  culture-specific  knowledge  within  any  culture  (Rasmussen  &  Sieck,  2012). 

Instructional  analysis  for  training  in  cultural  sensemaking  begins  with  Cognitive  Task  Analysis 
methods  to  identify  culturally  relevant  situations  for  inclusion  in  a  scenario-based  training  program 
(Chipman,  Schraagen,  &  Shalin,  2000).  The  specific  nature  of  the  resulting  scenarios  would  depend 
on  the  region  of  interest,  as  well  as  on  the  trainee's  job  (e.g.,  small  unit  leader  vs.  sociocultural 
analyst).  Curriculum  developers  then  use  CCA  to  characterize  both  native  decision  making  within 
these  challenging  situations  and  learner  expectations  regarding  the  native  decisions.  Specific 
knowledge-level  learning  objectives  then  result  from  comparing  the  learner  models  and  native 
models  to  identify  the  gaps  in  learners'  understanding  and  their  misconceptions  regarding  how 
natives  are  likely  to  decide. 

For  example,  in  the  application  to  Afghan  decision  making,  the  native  cognitive-cultural  model 
provided  the  target  concepts  for  training  U.S.  Marines  to  understand  Afghan  behavior.  New 
Marines  with  no  prior  deployments  made  open-ended  assessments  that  were  used  to  generate 
their  expected  models  of  the  Afghan  decisions  and  motivations.  The  program  developers  used  a 
coding  scheme  to  perform  a  quantitative  assessment  of  the  accuracy  of  these  target  learners' 
understanding  of  the  cultural  model  and  identify  critical  belief-value  relationships  that  they  either 
failed  to  perceive  or  had  misunderstood.  The  knowledge-level  (or  cognitive-level)  learning 
objectives  for  the  training  program  focused  on  closing  the  gaps  and  remedying  the  misconceptions 
revealed  through  this  assessment. 

The  cognitive  learning  objectives  in  the  example  above  are  culture-  and  job-specific,  as  is 
appropriate  for  trainees  who  must  focus  on  a  specific  region  for  a  year  or  more.  Training  in  cultural 
sensemaking  also  includes  culture-general  learning  or  metacognitive  strategies  that  build 
sensemaking  competence.  For  example,  in  the  Afghan-based  demonstration  project,  developers 
identified  learning  objectives  to  improve  information-seeking  strategies  by  comparing  the 
questions  that  the  target  learners  asked  to  better  understand  the  situation  to  the  questions  that 
expert  cultural  sensemakers  tend  to  ask  in  order  to  create  deep  understanding  (Sieck,  Smith,  & 
Rasmussen,  2008).  Experts  ask  cultural  sensemaking  questions  to  obtain  deeper  insight  into  the 
belief-value  relationships  relevant  to  explaining  and  anticipating  behavior.  Generally,  such 
questions  take  the  form  of  why,  why  not,  how,  what  if,  etc.  (Graesser,  Baggett,  &  Williams,  1996). 

The  framework  for  training  in  cultural  sensemaking  describes  a  process  through  which  culture- 
specific  learning  can  contribute  to  culture-general  competence.  The  underlying  concept  is  to 
provide  learners  with  baseline  cognitive-level  knowledge  of  the  factors  that  influence  the  decision 
making  of  culturally  different  others  in  specific  situations,  along  with  the  metacognitive  strategies 
that  enable  the  learners  to  progress  further  by  building  on  that  initial,  informed  understanding. 
This  should  enable  trainees  to  learn  more  efficiently  from  the  complex,  real-life  cultural  situations 
they  will  encounter  on  the  job,  and  further  expand  their  storehouse  of  experiences.  Applied 
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research  has  tested  the  principles  of  this  training  approach,  but  future  work  is  needed  to  develop  a 
full  training  system  based  on  the  approach  and  test  its  efficacy  at  supporting  sociocultural 
forecasting. 

5.2.  DoD's  W-ICEWS  program 

DoD's  W-ICEWS  program  focused  primarily  on  developing  and  testing  computational  models  to 
anticipate  and  understand  instability  and  violent  political  conflict  (Kettler  &  Hoffman,  2012; 
O'Brien,  2010).  The  developers  envisioned  a  system  that  could  provide  military  commanders  with 
predictions  as  to  which  countries  would  most  likely  experience  domestic  and  international  crises, 
with  forecasts  ranging  from  the  short  to  the  long  term.  Teams  in  the  W-ICEWS  program  developed 
competing  models  to  predict  historical  cases  of  instability  using  data  from  news  and  other  country 
background  information. 

DARPA  designed  W-ICEWS  to  enable  the  comparison  and  evaluation  of  several  different  forecasting 
approaches.  A  team  led  by  Lockheed  Martin-Advanced  Technology  Laboratories  (LM-ATL) 
developed  the  system  that  generated  the  most  accurate  forecasts,  making  correct  predictions 
about  80  percent  of  the  time  in  Phase  1,  and  improving  to  around  95  percent  accuracy  in  Phases  2 
and  3  (Kettler  &  Hoffman,  2012).  LM-ATL  used  Bayesian  methods  to  produce  forecasts  by 
integrating  a  few  distinct  kinds  of  modeling  systems,  such  as  agent-based  models,  logistic 
regression  models,  and  geospatial  network  models.  The  models  took  a  range  of  factors  into 
account,  including  sociocultural  information  such  as  ethnic-political  identities,  social  similarity 
profiles,  authority  structures,  trade  ties,  flow  of  people,  and  geographic  organization.  Interestingly, 
researchers  found  that  the  system  predicted  few  cases  of  civil  war  at  probability  levels  above  50%, 
and  some  of  the  models  apparently  never  generated  probabilities  greater  than  30%  (O'Brien, 
2010). 

To  determine  the  possible  training  implications  of  this  research  and  development  effort,  consider 
that  models  of  this  type  would  likely  serve  as  inputs  to  an  analyst  or  planner  responsible  for 
informing  a  command  team  about  sociocultural  events  of  interest.  In  such  cases,  the  analyst  should 
have  a  conceptual  understanding  of  the  models  and  the  process  by  which  they  arrive  at  their 
predictions.  Understanding  the  concepts  underlying  model  aggregation,  and  especially  the  reasons 
why  aggregated,  hybrid  models  often  outperform  others  would  also  be  extremely  useful.  This  level 
of  understanding  is  essential  to  ensure  that  operators  trust  and  incorporate  the  forecasting  system 
predictions  into  their  briefings  and  recommendations  (cf.  Wedgwood,  Ruvinsky,  &  Siedlecki,  2012). 

Beyond  this,  operators  need  to  understand  the  range  of  predictions  produced  by  the  systems: 
specifically,  how  to  interpret  low  probabilities  and  consider  base-rate  information  (overall 
proportions  of  times  that  events  occur).  A  sizeable  body  of  research  shows  that  superior  forecast 
performance  results  from  giving  substantial  weight  to  base-rates,  and  that  computational  models 
using  formalized  estimation  make  optimal  use  of  the  base-rate  information.  Models  that  produce 
only  slight  adjustments  from  base-rates  do  so  because  the  specific  factors  they  rely  on  are  only 
weakly  informative  as  to  the  events  being  predicted.  This  occurs  because  few  organizations 
currently  have  access  to  sociocultural  information  that  discriminates  much  beyond  base-rate 
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predictions.  Analysts  and  operators  must  learn  to  appreciate  that  the  specific  sociocultural 
information  they  have  on  hand  is  most  likely  not  as  discriminating  as  it  intuitively  appears. 

5.3.  lARPA's  ACE  Program 

lARPA's  ACE  program  on  contingent  forecasting  of  international  actor  reactions  to  possible  U.S.  and 
third-party  actions  has  also  undertaken  efforts  to  support  training  in  applied  forecasting.  According 
to  IARPA  (2013),  the  project  uses  crowdsourcing  techniques  to  "dramatically  enhance  the  accuracy, 
precision,  and  timeliness  of  forecasts  for  a  broad  range  of  event  types,  through  the  development  of 
advanced  techniques  that  elicit,  weight,  and  combine  the  judgments  of  many  intelligence  analysts." 
An  example  forecast  problem  is:  "Will  Iran  and  the  United  States  commence  official  nuclear 
program  talks  before  1  April  2013?"  The  resulting  forecasts  span  a  global  domain,  from  the 
outcome  of  presidential  elections  in  Taiwan  to  the  potential  of  a  downgrade  of  Greek  sovereign 
debt,  which  presents  interesting  challenges  in  terms  of  bringing  relevant  regional  and  cultural 
knowledge  to  bear.  IARPA  evaluates  ACE  predictions  using  the  widely  accepted  Brier  score,  or 
average  sum  of  squared  differences  between  probability  forecasts  and  the  actual  outcomes,  as 
described  previously  (Brier,  1950). 

With  respect  to  training,  one  project  site  employs  a  "Forecasting  Ace  University"  that  provides 
detailed  information  on  specific  aspects  of  forecasting,  along  with  a  set  of  forecast  training 
modules  (Warnaar  et  al.,  2012).  The  training  topics  include  the  scoring  rules  used  by  the  site,  a 
description  of  high-stakes  forecasting,  calibration  of  forecasts,  the  use  of  base-rates  to  alleviate 
over-prediction,  and  ways  of  evaluating  the  credibility  of  information  sources  to  promote  better 
discrimination.  The  system  delivers  computer-based  training  (CBT)  modules  on  several  topics  over 
the  web  (i.e.,  e-learning).  Each  module  includes  multimedia  presentations  and  interactive  exercises 
along  with  explanation  feedback  to  aid  concept  acquisition  and  application.  Site  users  also  receive 
information  and  feedback  relevant  to  specific  forecasts,  such  as  information  about  others' 
forecasts  and  rationales  (environmental  feedback),  trends  in  crowd  forecasts  over  time,  and 
information  about  their  own  forecast  accuracy  (performance  feedback). 

Another  team  has  also  sought  to  determine  how  best  to  train  and  support  forecasters,  and  has 
reported  demonstrated  gains  due  to  training  (Ungar,  Mellers,  Satopaa,  Tetlock,  &  Baron,  2012). 
This  team  initially  expressed  some  skepticism  about  the  possible  benefits  of  allowing  individual 
forecasters  to  share  information  (a  form  of  environmental  feedback)  or  to  receive  training  in  the 
use  of  probabilities,  including  performance  types  of  feedback.  For  example,  they  noted  that 
permitting  experts  to  share  information  about  their  predictions  and  reasoning  had  the  same 
potential  of  leading  to  "group-think"  as  of  leading  to  better  models  of  the  phenomenon.  On  the 
probability  training  side,  they  noted  that  conceptual  knowledge  of  probability  and  statistics  does 
not  disable  forecasters'  natural  intuitions  and  use  of  flawed  heuristics  when  making  predictions. 
They  also  pointed  out  that  systematic  cognitive  biases,  such  as  overconfidence,  might  be  more 
readily  corrected  using  mathematical  transformations  during  the  aggregation  of  forecasts  across 
members  of  the  "crowd." 

Despite  their  initial  pessimism,  the  team  experimentally  tested  the  effects  in  a  controlled  fashion. 
Conditions  within  the  experiment  included  probability  training  and  no  training,  as  well  as 
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forecasters  who  worked  independently  and  those  who  were  assigned  to  teams  of  15  to  20 
members  who  consulted  with  one  another  over  each  forecast  (the  full  design  included  several 
other  conditions  as  well).  Their  primary  findings  included  that  the  team  condition,  which  enables 
environmental  feedback  in  the  form  of  sharing  region-specific  and  other  information  related  to 
each  forecast,  significantly  outperformed  the  independent  forecaster  conditions.  In  addition,  the 
researchers  found  probability  training  was  beneficial,  and  indeed  augmented  the  benefits  of  the 
team  condition,  such  that  the  probability  training  proved  most  effective  for  people  who  made 
forecasts  within  team  environments. 

Another  interesting  aspect  of  W-ICEWS  and  ACE  is  that  both  directly  embed  training  as  a  core 
component  of  the  forecasting  system.  Researchers  should  explore  additional  development 
approaches  along  the  lines  of  integrating  forecast  systems  and  forecast  training  systems. 


6.  Advancing  Training  in  Sociocultural  Forecasting 

The  efforts  surveyed  above  reveal  several  science  and  technology  gaps  that  researchers  should 
address  to  move  this  capability  area  toward  operational  usage.  This  particular  sub-area  of 
sociocultural  sensemaking  is  perhaps  less  developed  than  others,  so  the  following  discussion 
primarily  addresses  topics  that  applied  research  should  resolve,  such  as  potential  skills,  knowledge, 
and  abilities  that  should  be  tested  as  training  requirements.  However,  this  section  also  contains 
some  suggestions  as  to  operational  training  system  requirements. 

As  noted  previously,  studies  on  training  for  probabilistic  forecasting  indicate  that  knowledge  of  the 
culture  or  area  does  not  by  itself  enable  analysts  to  accurately  anticipate  future  events; 
performance  feedback  using  probability  accuracy  measures,  such  as  calibration,  is  also  needed. 
Applied  research  with  professionals  in  the  sociocultural  domain  should  test  this  proposition.  If  it 
holds,  then  training  systems  that  improve  calibration  of  sociocultural  forecasts  could  potentially 
lead  to  immediate  gains  in  both  the  forecasting  process  and  forecast  accuracy  among  these 
experts. 

Fundamental  cultural  values  that  underlie  long-term  changes  to  societies  should  receive  greater 
attention  in  applied  research  and  development,  as  they  influence  current  events  and  future 
prospects  within  areas  of  interest.  Along  these  lines,  applied  research  should  test  the  potential 
value  of  using  a  small  set  of  sociocultural  factors  directly  related  to  cultural  change  as  general 
predictors  for  problems  in  the  sociocultural  sphere.  Research  should  also  test  the  hypothesis  that 
analysts  could  be  readily  trained  to  recognize  these  factors,  and  that  this  would  aid  in  preventing 
over-prediction  and  enhancing  calibration  of  sociocultural  forecasts.  In  addition,  research  should 
test  the  hypothesis  that  training  to  consider  this  specialized  set  of  factors  can  improve 
understanding  of  why  cultural  change  frequently  does  not  happen  or  produces  only  temporary 
effects.  Similarly,  researchers  should  direct  more  efforts  toward  fine-grained  analyses  to 
understand  the  comparative  predictive  value  of  specific  sociocultural  factors  typically  used  in 
current  sociocultural  models,  and  to  determine  new  measures  that  yield  more  discriminative 
information.  Uncovering  new  measures  will  likely  be  best  served  by  precise  and  persistent  sensing 
of  regions  of  interest. 
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Researchers  should  examine  the  potential  utility  of  training  analysts  to  routinely  and  intuitively 
represent  sociocultural  forecast  problems  at  the  four  distinct  levels  of  analysis  (ecological- 
historical,  sociopolitical  institutions,  everyday  social  situations,  and  psychological  adaptations) 
identified  by  Gelfand  et  al.  (2011).  Such  research  would  test  the  important  hypothesis  that  intuitive 
understanding  of  the  interactions  among  these  levels,  which  mutually  support  the  status  quo,  can 
improve  sociocultural  forecasting  accuracy,  especially  in  terms  of  calibration. 

Research  should  seek  to  verify  whether  cognitive-cultural  models  actually  provide  a  framework 
that  supports  forecasting  by  enabling  analysts  to  reference  a  cultural-insider  point  of  view,  and 
explore  several  additional  hypotheses  associated  with  the  potential  benefits  of  such  models.  For 
example,  one  hypothesized  benefit  is  that  the  form  of  the  model  encourages  thinking  about 
specific  elements  of  information  pertinent  to  forecasts,  and  hence  aids  novice  analysts  to  focus 
their  efforts  on  acquiring  specific  cultural  information  relevant  to  their  forecast  problems.  Another 
is  that  inclusion  of  quantitative  estimates  of  cultural-idea  prevalence  could  improve  both 
discrimination  and  calibration.  Applied  research  efforts  are  needed  to  test  this  claim,  as  well  as  to 
examine  the  training  required  to  enable  use  of  cognitive-cultural  models.  Such  training  could  allow 
analysts  to  comprehend  these  models,  as  well  as  to  create  them  directly  from  the  source  materials 
they  are  working  with,  as  a  way  to  take  a  cultural  other's  perspective. 

A  fair  amount  of  applied  research  has  sought  to  validate  the  principles  underlying  the  approach  to 
training  analysts  in  cultural  sensemaking,  and  has  shown  promising  results.  Future  work  should  aim 
to  develop  a  full  CBT  system  based  on  the  approach  and  should  test  its  efficacy  at  supporting 
sociocultural  forecasting.  A  lightweight  e-learning  system  that  relies  on  standard  multimedia 
presentation  capabilities  combined  with  interactive  exercises  would  likely  offer  a  reasonable  near- 
term  solution.  However,  systems  that  incorporate  interactive  tutors  using  artificial  intelligence  may 
better  identify  and  adapt  to  the  wide  variety  of  possible  background  knowledge  and  attitudes  that 
learners  bring  to  the  cultural  training  environment  (Woolf,  2009). 

Research  programs  should  explore  training  that  might  improve  operators'  conceptual 
understanding  of  various  formal  sociocultural  models  (e.g.,  logistic  regression,  agent-based  models, 
hybrid  models)  and  the  processes  by  which  they  arrive  at  their  predictions.  Understanding  the 
concepts  underlying  these  models  and  the  nature  of  the  benefits  of  aggregation  across  models 
would  help  improve  operator  trust  and  adoption  of  the  forecasting  systems.  A  related  line  of 
applied  research  would  investigate  training  to  understand  the  current  limits  of  sociocultural 
prediction. 

Existing  training  efforts  provide  encouraging  support  to  the  idea  that  probability-oriented  training 
improves  specific  types  of  sociocultural  forecasts.  The  findings  should  be  extended  to  other  types 
of  sociocultural  forecasting.  In  addition,  some  programs  have  succeeded  in  integrating  forecast 
training  with  forecasting  support  systems;  further  development  efforts  should  capitalize  on  this 
initial  success. 

As  stated  at  the  outset,  all  areas  of  science  struggle  to  meet  the  challenge  of  predicting  events. 
Research  and  development  efforts  that  address  sociocultural  sensemaking,  particularly  in  the  areas 
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of  modeling  and  simulation,  have  made  important  advances  in  addressing  this  challenge  in  the 
cognitive  science  of  culture.  Future  efforts  to  fill  the  gaps  and  test  the  hypotheses  described  here 
will  further  enhance  the  capability  to  train  forecasters  working  in  this  domain,  and  can  thereby 
improve  predictions  in  operational  settings. 
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Section  Four:  Mitigating 

Mitigation  of  behaviors  in  support  of  operational 
objectives 

Gary  L.  Klein,  The  MITRE  Corporation 


The  capabilities  described  in  the  previous  sections  provide  a  foundation  for  the  final  step  in  the 
operational  cycle:  mitigating  the  influence  of  adverse  sociocultural  behavior  in  the  conduct  of  a 
mission.  To  accomplish  this,  commanders  must  develop,  prioritize,  execute,  and  measure  courses 
of  action  (COAs)  grounded  in  the  social  and  behavioral  sciences  (Schmorrow,  2011).  This  mitigation 
capability  builds  on  all  the  foregoing  ones,  and  should  begin  the  cycle  anew  by  providing  new 
information  that  updates  U.S.  forces'  understanding  of  the  sociocultural  behavior  terrain. 
Projecting  the  impact  of  sociocultural  behavior  interacting  with  COAs  requires  the  use  of  models 
for  robust  decision  making,  for  strategic-level  sociocultural  theory,  for  systems  integration,  for 
decision  space  visualization,  and  for  agile  data  collection.  Moreover,  operating  these  new  models 
and  their  applications  requires  new  kinds  of  training. 

Making  robust  mitigation  decisions  in  today's  complex,  uncertain  environment  demands  more  than 
situation  awareness.  Decision  makers  must  achieve  option  awareness  (Pfaff,  Klein,  Drury,  Moon, 
Liu,  &  Entezari,  2012).  Option  awareness  level  1  means  recognizing  which  option  will  yield  the  most 
satisfactory  results  over  the  widest  swath  of  plausible  futures.  Therefore,  this  level  requires  an 
ability  to  compare  the  landscapes  of  plausible  outcomes  that  may  occur  for  each  COA  under  a 
variety  of  conditions.  Option  awareness  level  2  means  comprehending  which  factors  to  facilitate  or 
mitigate  in  order  to  increase  the  likelihood  of  achieving  satisfactory  outcomes.  Hence,  this  level 
requires  the  ability  to  "drill  down"  to  discern  the  sociocultural  factors  and  environmental 
conditions  that  interact  with  each  COA  to  produce  better  (or  worse)  outcomes.  Finally,  option 
awareness  level  3  requires  the  ability  to  map  the  factors  discovered  through  analysis  back  to  the 
real  world  so  that  commanders  can  conceive  the  real-world  actions  needed  actually  to  facilitate  or 
mitigate  those  factors.  Supporting  option  awareness  calls  for  unique  applications  of  modeling  to 
generate  multi-dimensional  decision  spaces,  data  processing  to  feed  those  models,  visualization  to 
make  those  decision  spaces  comprehensible,  and  training  to  engage  in  this  sophisticated  level  of 
development  and  analysis.  Each  chapter  in  this  section  describes  how  its  technology  area  applies  to 
these  endeavors. 

The  chapter  on  computational  modeling  examines  a  sampling  of  computational  approaches  that 
may  facilitate  COA  development,  analysis,  and  comparison,  noting  the  potential  for  positive  impact 
on  decision  making  for  mitigation.  Even  with  the  visualization,  data  processing,  and  modeling 
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techniques  described  in  previous  chapters,  no  human  analyst,  no  matter  how  talented  or 
knowledgeable,  can  manage  the  amount  and  complexity  of  the  information  that  must  be  weighed 
for  mitigation  decisions.  In  this  chapter,  the  authors  discuss  how  computational  modeling  can 
relieve  some  of  this  burden,  allowing  decision  makers  to  synthesize  and  integrate  the  models, 
knowledge,  and  insights  from  the  understand,  detect,  and  forecast  phases  of  analysis  to  provide 
better  operational  awareness.  Using  many  methods  similar  to  the  big-data  analytics  employed  by 
corporations  such  as  Amazon  or  Google,  analysts  can  exploit  this  wealth  of  sociocultural  knowledge 
to  support  decision  making  and  determine  the  most  robust  COA  in  many  critical  situations. 

Because  decision  making  for  mitigation  relies  so  heavily  on  computational  modeling,  data 
processing  for  the  mitigation  capability  must  support  interfaces  among  disparate  models  - 
interfaces  so  complicated  that  their  design  is  itself  an  act  of  modeling.  Consequently,  the  chapter 
on  data  processing  centers  on  the  representation  of  the  data  that  are  input,  output,  and  traded 
between  models  and  the  interfaces  among  the  models  that  arrange  how  those  data  are  traded  and 
are  translated  into  the  languages  of  the  different  models.  Applied  to  COA  analysis  in  sociocultural 
scenarios,  data  representations  span  strategy  and  narratives.  They  also  include  co-evolutionary 
model  interfaces,  game  theoretical  arrangements  of  disparate  models,  and  interfaces  that  preserve 
uncertainty.  This  chapter  first  presents  an  example  of  an  irregular  warfare  battlespace,  and  then 
discusses  state-of-the-art  social  data  representations  and  interfaces  of  social  data  for  COA  analysis 
of  irregular  warfare,  giving  examples  of  their  value  in  representing  social  phenomena. 

At  its  best,  effective  visualization  can  lead  to  a  deeper  understanding  of  COA  robustness,  resulting 
in  better  outcomes  in  more  situations.  The  chapter  on  visualization  covers  the  underpinnings  of 
effectively  using  visualization  to  help  decision  makers  deeply  comprehend  their  options.  The 
author  discusses  the  research  to  date  in  this  area  and  promising  directions  for  future  research  in 
this  nascent  area  of  using  interactive  visualization  to  help  compare  and  contrast  COAs.  Most 
visualization  tools  today  present  situation  awareness  data,  leaving  the  burden  on  the  user  to 
extrapolate  from  the  details  of  a  situation  to  decide  on  a  good  COA.  This  chapter  describes  new 
interactive  visualization  tools  that  support  each  level  of  option  awareness,  giving  decision  makers  a 
much  broader  and  deeper  view  of  their  options.  If  the  visualization  enables  option  awareness  level 
1,  decision  makers  will  more  rapidly  see  and  understand  the  comparative  robustness  of  their 
options.  If  the  visualization  enables  option  awareness  level  2,  the  decision  makers  will  comprehend 
the  key  trade-offs  between  those  options  and  the  factors  that  make  one  option  more  robust  than 
another.  Building  on  levels  1  and  2,  decision  makers  who  achieve  option  awareness  level  3  can 
conceive  new,  more  effective  options  based  on  a  deep  understanding  of  relative  robustness  and 
the  factors  driving  that  robustness. 

As  additional  technologies  to  support  sociocultural  mitigation  improve  and  are  deployed,  more 
operational  users  will  need  training  on  how  to  use  these  tools,  data,  and  analytics  appropriately 
and  optimally  to  mitigate  and  shape  potential  futures.  In  contrast,  limited  time,  tools,  and  capacity 
currently  restrict  the  number  of  prospective  COAs  for  mitigation  that  operational  planners  can 
generate  and  assess,  because  the  processes  implemented  today  are  manually  intensive  and  seldom 
integrated  (computationally)  with  sociocultural  knowledge.  The  quality  and  kinds  of  information 
that  new  technologies  make  available  to  users  and  the  potential  utility  of  this  content  will  begin  to 
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blur  the  traditional  "lanes"  separating  current  career  fields  such  as  intelligence,  operations 
research,  and  operational  planning.  This,  in  turn,  will  require  innovation  in  training  to  prepare  users 
to  effectively  recognize  and  leverage  relevant  content  in  order  to  analyze  COAs  for  mitigation.  For 
example,  training  on  model  interpretation,  typically  concentrated  in  a  particular  career  field  such 
as  operations  research,  must  be  expanded  to  ensure  all  operational  users  can  use  emerging 
mitigation  technology  rapidly  and  properly.  The  chapter  on  training  focuses  on  defining  the  skills 
(abilities  necessary  to  perform  the  task)  and  knowledge  (facts,  concepts,  and  principles  needed  to 
perform  the  task)  for  mitigation  decision  making  in  order  to  establish  new  guidelines  for  training. 
An  initial  analysis  of  the  functional  tasks  involved  in  mitigation  highlights  several  key  training 
considerations,  such  as  appropriate  objectives  and  ideal  training  formats  (e.g.,  computer-based 
training,  classroom,  or  exercise). 
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1.  Introduction 

Data  processing  for  course  of  action  (COA)  analysis  of  irregular  warfare  (IW)  scenarios  differs  from 
COA  analysis  of  conventional  warfare  scenarios  for  many  of  the  same  reasons  that  social  science 
differs  from  physics.  A  significant  difference  is  that  physics  models  are  models  of  things,  while 
social  models  are  models  of  modelers  because  all  people  model  their  world.  IW  focuses  on  the 
reactions  of  a  population  of  these  human  modelers— reactions  that,  unlike  physics-based 
outcomes,  are  neither  straightforward  nor  equation-based.  This  distinction  encapsulates  the 
challenge  of  processing  sociocultural  data.  Sociocultural  data  processing  requires  interfaces 
between  disparate  models— interfaces  so  complicated  that  designing  them  is  itself  an  act  of 
modeling.  Physics  models,  by  contrast,  typically  implement  Newtonian-based,  validated  data  that 
can  be  held  in  limited  repositories  and  require  no  translation  because  the  models  rely  on  shared 
and  accepted  concepts. 

IW  modeling  involves  reproducing  a  series  of  moves  and  countermoves  that  antagonists  might  use 
to  convince  populations  to  model  and  act  upon  events  in  one  way  and  not  another.  How  agents— 
whether  human  or  software  agents— react  to  IW  moves  results  from  their  interpretations,  or  the 
way  that  they  model  the  moves.  Information  operations  (10)  present  narratives  to  counter  terrorist 
narratives;  Civil  Military  Operations  (CMO)  embody  these  counternarratives.  Both  terrorist  and 
CMO  narratives  are  processed  by  minds  that  create  interpretive  models  to  determine  which 
narratives  make  sense  to  them.  A  population's  reaction  to  IW,  10,  or  CMO  is  represented  by  the 
computation  in  and  the  interfacing  of  models  in  individual  agents'  "minds." 

Consequently,  data  processing  for  mitigation  encompasses  the  representation  of  the  data  input, 
output,  and  trades  between  models,  the  interfaces  between  the  models  that  arrange  how  those 
data  are  traded,  and  the  ways  in  which  the  data  are  translated  into  the  different  languages  of  the 
different  models.  Applied  to  COA  analysis  in  sociocultural  scenarios,  data  representations  include 
representations  of  strategy  and  narratives.  They  also  include  coevolutionary  model  interfaces, 
game  theoretical  arrangements  of  disparate  models,  and  interfaces  that  preserve  uncertainty. 
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This  chapter  uses  the  film  The  Battle  of  Algiers  as  an  example  to  provide  context  for  the  IW 
battlespace.  It  then  discusses  state-of-the-art  social  data  representations  and  interfaces  among 
social  models  for  COA  analysis  of  IW,  giving  examples  of  their  value  in  the  representation  of  social 
phenomena. 


2.  IW  Strategic  Scenario 

To  design  data  representations  for  COA  analysis  with  sociocultural  models,  we  must  first 
understand  how  the  models  are  used.  In  the  context  of  this  chapter,  the  moves  and  countermoves 
of  IW  form  the  crux  of  what  sociocultural  COA  analysis  examines.  These  moves  and  countermoves 
represent  the  antagonists'  attempts  to  change  the  population's  perceptions.  To  anchor  the 
discussion  of  sociocultural  data  for  COA  analysis,  we  present  a  synopsis  of  the  movie  The  Battle  of 
Algiers,  produced  and  written  by  the  actual  insurgents  and  shown  in  the  Pentagon  as  an  archetypal 
IW  scenario  in  which  an  insurgent  group  used  IW  tactics  to  make  the  transition  from  unpopular  to 
victorious  (Solinas,  Yacef,  Pontecorvo,  Musu,  Haggiag,  &  Martin,  1976). 

In  1954,  the  French  suffered  defeat  at  Dien  Bien  Phu,  Vietnam,  prompting  French  withdrawal  from 
its  Indochinese  colonies.  The  French  now  appeared  vulnerable,  and  Arab  resistance  elements  in 
Algeria  believed  that  the  French  would  not  want  to  engage  in  another  colonial  war.  However,  the 
French  considered  Algeria  fully  a  part  of  France  and  were  determined  to  counter  any  rebellion. 
Even  so,  the  French  public  was  susceptible  to  narratives  that  highlighted  the  political  and  economic 
discrimination  that  indigenous  Algerians  suffered.  The  regime  deprived  10  million  indigenous  Arabs 
in  Algeria  of  political  rights,  and  almost  all  of  the  economy  was  in  the  hands  of  one  million  French 
Algerians,  who  enjoyed  better  houses  and  schools  than  the  Arabs.  Note  that  the  points  of 
vulnerability  of  the  French,  and  the  sources  of  strength  for  the  Arabs,  were  cognitive:  the  war¬ 
weariness  of  the  European  French,  the  European  French  conviction  that  Algeria  was  an  integral 
part  of  France,  and  the  potential  for  the  indigenous  Algerians  to  perceive  injustice  were  all  ideas 
critical  to  motivating  the  popular  will.  IW  simulations  should  capture  such  ideas,  how  they  spread, 
and  how  they  motivate  IW  actions. 

The  Algerian  insurgency,  known  as  the  National  Liberation  Front  (FLN),  argued  that  the  French 
Algerians  did  not  respect  their  culture  and  incited  decadent  behavior  such  as  prostitution  and 
drunkenness.  The  FLN  therefore  campaigned  to  purify  Algeria,  imposing  their  own  death  sentences 
on  the  population  for  behavior  that  contradicted  Muslim  values.  The  movement  assumed  other 
functions  of  government  as  well,  such  as  performing  wedding  ceremonies.  The  FLN  strategy  was  to 
win  the  respect  and  support  of  the  people  and  thereby  secure  hideouts  for  themselves.  Ethnic 
violence  between  the  French  and  Arab  Algerians  included  an  uprising  in  1945  in  which  90  French 
Algerians  were  killed,  followed  by  a  French  massacre  of  over  40,000  Arabs.  Many  Algerian  Arab 
insurgents  cited  this  massacre  as  a  factor  in  their  decision  to  join  a  terrorist  group.  Despite  this 
history,  in  1954  every  newspaper  in  Algeria,  including  the  communist  papers,  favored  elimination 
of  the  FLN.  Note  that  the  massacre  constituted  evidence  that  countered  the  newspapers'  anti¬ 
insurgency  narrative,  while  the  contrast  between  French  behavior  and  Muslim  values  supported 
the  narrative  that  insurgents  who  suppressed  these  behaviors  deserved  respect. 
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When  the  French  guillotined  FLN  leaders  in  1954,  the  FLN  complained  that  they  were  being  treated 
as  criminals  rather  than  combatants,  and  organized  an  insurrection  against  the  Algerian  police.  The 
police  reacted  by  instituting  curfews,  checkpoints,  and  searches  of  Arabs,  sometimes  violating 
cultural  restrictions  on  searching  women.  The  French  Algerians,  together  with  members  of  the 
police,  also  bombed  the  Kasbah,  a  concentrated  ethnic  Arab  section  of  Algiers,  killing  75  Arab 
civilians.  The  Arab  populace  rushed  out  to  demonstrate  in  the  streets,  but  the  FLN  stopped  the 
demonstrations,  telling  the  people  that  they  would  be  slaughtered  and  that  the  FLN  would  avenge 
their  compatriots.  The  FLN  did  this  because  they  feared  that  widespread  killing  would  intimidate 
the  people  whom  they  hoped  to  rally  to  their  side.  Note  that  protecting  the  populace  and  avenging 
them  supported  the  narrative  of  the  FLN's  legitimacy  as  an  ally  and  prevented  emergence  of  a 
discouraging  narrative  of  the  hopelessness  of  standing  up  to  the  French. 

In  1956,  the  FLN  avenged  the  Kasbah  bombing  by  launching  a  simultaneous  attack  on  three  French 
civilian  targets,  killing  many  French  Algerians.  Some  French  Algerians  responded  by  targeting 
innocent  Arabs  and  even  Arab  children.  The  government  of  Algiers  turned  the  situation  over  to  the 
French  army. 

In  1957,  the  FLN  coordinated  an  eight-day  general  strike  to  gain  the  attention  and  support  of  the 
international  community,  specifically  the  United  Nations  (UN).  Despite  widespread  support  within 
Algeria,  the  strike  had  little  impact  on  the  UN  because  the  French  army  broke  the  strike  in  a  day  by 
forcing  people  out  of  their  homes,  opening  the  shops  themselves,  and  compelling  owners  to  open 
their  stores  when  they  tried  to  protect  their  goods.  Flowever,  the  government's  aggressive 
response  served  to  anger  the  Arab  population  further.  The  FLN's  terrorist  actions  and  the  French 
retaliation  led  more  and  more  people  to  believe  the  narrative  that  the  French  were  the  enemies  of 
indigenous  Algerians.  The  insurgents  successfully  turned  the  Algerian  Arabs  and  the  international 
community  against  the  French  government  by  inciting  harsh  reactions.  At  the  same  time  they 
insulated  the  people  and  themselves  from  intimidation  by  timing  demonstrations  and  strikes  so 
that  supporters  would  feel  strength  in  numbers  and  not  fear  reprisal. 

Both  sides  paid  attention  to  the  perceptions  of  the  people  and  the  international  community  by 
following  unwritten  rules  of  legitimate  action,  evidenced  by  their  seeking  to  justify  their  actions. 
For  instance,  the  insurgents  sought  favorable  opinions  by  advancing  the  theme  that  the  French 
government  had  crossed  the  line  by  harming  civilians.  The  insurgents  also  presented  the  Algerian 
government  with  "damned  if  you  do,  damned  if  you  don't"  situations,  where  the  cure  had  the  side 
effect  of  causing  more  damage.  For  instance,  by  pressing  the  government  to  end  checkpoints  the 
insurgents  would  benefit  no  matter  how  the  government  responded:  they  would  reap  direct  gains 
if  the  checkpoints  ceased,  but  would  obtain  additional  popular  support  by  making  claims  about 
humiliating  searches  and  violations  of  custom  if  the  checkpoints  continued. 

Although  the  local  population  generally  favored  the  insurgents  and  sheltered  them,  the  Algerian 
government  and  the  French  army  achieved  short-term  tactical  success  by  arresting  and  often 
eliminating  FLN  members.  The  FLN  instructed  its  adherents  to  remain  silent  for  24  hours  if  they 
were  captured,  which  would  render  their  knowledge  of  safe  houses  useless.  To  counter  these 
instructions,  the  French  army  used  brutal  torture  techniques  to  obtain  information  in  the  few  hours 
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during  which  it  was  still  useful.  However,  the  army  weakened  its  own  cause  by  torturing  some 
persons  who  were  not  insurgents.  This  further  deepened  the  belief  that  the  French  were  the 
enemy  of  the  indigenous  Algerians.  Over  time,  the  Arab  population  and  the  French  press  learned 
about  the  torture,  giving  rise  to  an  even  larger  insurgency  in  1960.  The  result  was  a  strategic  defeat 
for  the  French-supported  Algerian  government,  even  though  the  French  had  "won  the  battle"  by 
eliminating  nearly  all  insurgents.  The  strategic  defeat  caused  by  the  Algerian  populace's  changing 
beliefs  ultimately  led  to  the  end  of  French  control  in  Algeria. 


3.  Representing  IW  Strategic  Data 

In  the  Battle  of  Algiers  scenario,  insurgent  moves  embodied  strategies  to  make  the  French  act  in  a 
manner  that  advanced  the  narrative  of  "the  French  as  oppressors."  Although  the  French  lacked  a 
counternarrative,  modern  doctrine  combines  10  with  CMO  to  support  narratives  reinforcing  the 
legitimacy  of  the  host  nation  or  friendly  forces.  To  represent  insurgent  and  coalition  force  COAs  for 
an  IW  scenario,  modelers  must  represent  the  narratives,  the  population's  reactions  to  the 
narratives,  and  events  surrounding  the  narratives.  Additionally,  they  must  capture  the  insurgents' 
mental  model  of  the  population  and  the  coalition  forces'  mental  model  of  the  population,  if 
applicable,  as  well  as  the  reactions  of  insurgents  and  coalition  forces  to  each  other.  Steve  Corman's 
project  on  Identifying  Terrorist  Narratives  and  Counter-Narratives  (ITNC)  at  the  Center  for  Strategic 
Communication  and  Arizona  State  University,  John  Horgan's  project  on  Competitive  Adaptation  in 
Terrorist  Networks  (CATNet)  at  Pennsylvania  State  University  (Horgan,  Horne,  Vining,  Carley, 
Bigrigg,  Bloom,  &  Braddock,  2011),  and  Jerzy  Rozenblit's  Asymmetric  Threat  Response  And  Analysis 
Program  (ATRAP)  at  the  University  of  Arizona  (Mitchell,  Craw,  &  Ten  Eyck,  2011)  present  innovative 
ways  to  represent  strategic  ideological  moves  and  countermoves. 

According  to  narrative  paradigm  theory,  people  think  in  stories  (or  frames)  and  interpret  events 
through  these  frames.  Corman,  Ruston,  &  Fisk  (2012)  developed  a  method  to  extract  frames  from 
text  that  represent  cultural  narratives  and  cluster  them  into  groups  of  "master  narratives"  that 
contain  archetypes.  Corman  uses  the  Natural  Language  Processing  (NLP)  technique  to  identify 
similar  stories  and  concepts  in  these  stories  as  general  archetypes,  and  then  counts  how  often 
these  archetypes  appear  in  news  stories  to  obtain  a  measure  of  belief  of  the  host  nation.  Corman 
examines  the  archetypal  master  narratives  for  evidence  that  friendly  forces  can  show  the  host 
nation  population  to  discredit  terrorist  narratives,  on  the  assumption  that  the  average  person  can 
be  convinced  with  logical  argument.  He  also  examines  possible  narratives  that  the  host  nation 
could  adopt  to  counter  the  terrorist  narrative,  thus  helping  the  host  nation  population  to  interpret 
U.S.  actions  as  we  wish. 

In  the  Algiers  example,  the  host  population  initially  internalized  a  narrative  that  they  were  French 
citizens,  accepting  blame  for  the  massacre  of  their  citizens  following  their  1945  uprising.  However, 
that  framing  of  the  situation  did  not  sit  well  with  the  indigenous  Algerians:  because  the  narrative 
minimized  the  disproportion  of  the  response  it  caused  cognitive  dissonance.  The  insurgents  took 
advantage  of  this  dissonance  by  offering  the  population  a  way  to  resolve  it  cognitively,  which  also 
put  the  FLN  in  power.  The  insurgents'  counter-narrative  focused  on  injustice,  framing  the  situation 
as  a  breach  of  contract,  as  well  as  one  of  moral  decadence,  if  Algeria  remained  under  French  rule. 
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This  theme  of  unjust  treatment  became  a  master  narrative  in  the  Muslim  world  that  has  persisted 
through  the  Arab  Spring  of  2010.  The  French  offered  only  the  counternarrative  that  individuals  who 
joined  the  resistance  would  be  tortured  and  the  population  massacred.  This  narrative  of  fear 
caused  revulsion  not  only  among  the  indigenous  population  but  also  in  France  proper,  and  was  not 
consonant  with  either  party's  value  system.  The  French  resolved  the  dissonance  by  withdrawing 
their  forces  and  granting  independence  to  Algeria. 

In  Corman's  unique  formalization  of  narratives,  an  analyst  starts  with  free  text  such  as  news 
articles,  parsed  with  resource  description  framework  (rdf)  triplets  of  subject,  verb,  and  object.  The 
analyst  identifies  stories  in  the  text,  creating  graphs  that  represent  the  stories.  After  manually 
dissecting  many  articles  in  this  manner,  the  analyst  clusters  individual  graphs  to  obtain  an  objective 
"story  distance"  measure.  The  analyst  measures  story  distance  by  taking  all  triplets  in  one  story  and 
comparing  them  to  all  triplets  in  another  story,  scoring  the  same  noun  or  its  synonym  with  its 
inverse  document  frequency  (IDF),  a  measure  of  the  noun's  uniqueness.  If  the  verb  is  the  same  or  a 
synonym  (as  determined  by  the  verbNet  thesaurus)  its  IDF  score  is  added  to  the  triplet  distance 
score.  The  sum  of  the  added  IDF  scores  of  all  the  triplets  determines  the  similarity  of  the  entire 
story  to  another  story.  Then,  the  analyst  assumes  that  close  stories  adhere  to  the  same  general 
archetype,  and  extracts  the  similar  nouns  and  verbs  to  make  up  a  general  story  pattern  and 
objectively  measure  the  frequency  of  that  pattern  in  news  articles  as  a  measure  of  belief.  For 
example,  this  technique  could  enable  analysts  to  measure  the  frequency  of  the  master  narrative  of 
injustice  in  the  Algerian  and  French  press  during  the  battle  of  Algiers,  mapping  the  population's 
beliefs  over  time. 

The  ability  to  model  10,  including  10  content,  the  impact  of  10  on  the  population,  and  the  reasons  a 
population  accepts  (or  does  not  accept)  10,  remains  the  largest  gap  in  IW  modeling.  Analysts  can 
use  Corman's  objective  formalization  of  stories  as  an  indicator  of  a  story's  recurrence  as  well  as  an 
explanation  of  why  a  message  is  accepted  or  not.  Modelers  can  use  this  formalization  to  represent 
the  cognitions  of  host  nation  agents  in  simulation  models,  and  to  feed  these  cognitions  into  the 
agents'  "minds"  directly  from  news  articles.  This  technique  not  only  allows  modelers  to  enter  data 
into  simulations  and  to  compare  simulation  data  against  real  world  data  for  validation,  but  also 
facilitates  a  representation  of  cognition  that  enables  models  to  compute  the  acceptance  or 
rejection  of  messages.  Corman's  method  lends  validity  to  studies  precisely  because  it  addresses  the 
causation  behind  message  acceptance  as  it  exists  in  cognition. 

By  contrast,  Florgan's  research  describes  terrorists'  evolving  behavior  throughout  a  conflict  or 
throughout  the  terrorist's  career.  Florgan  is  a  widely  acclaimed  expert  on  terrorism;  his  CATNet 
project  shapes  the  detailed  interviews  and  field  work  he  conducted  into  a  story  of  competitive 
adaption,  or  coevolution,  between  insurgencies  and  government.  Florgan  defines  competitive 
adaptation  as  the  ability  to  learn  from  an  adversary  and  gain  a  variety  of  skills  in  order  to  overcome 
the  enemy.  Florgan's  methodology  involves  detailed,  structured  interviews  to  capture  how 
individual  terrorists  learn  and  change  throughout  their  careers,  asking  questions  about  what  they 
learn  from  the  governments  they  are  fighting.  FHis  studies  revealed  that  terrorists  act  in  response  to 
their  government  enemies:  with  every  new  initiative  by  the  government,  the  terrorists  find  a  new 
strategy  to  circumvent  the  government  and  enforce  their  will  on  the  people.  Florgan's  interviews 
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capture  the  coevolutionary  dynamics  of  insurgencies  in  the  Military  Information  Support 
Operations  (MISO)  moves  and  countermoves  that  are  central  to  terrorist  activities.  The 
coevolutionary  information  that  Horgan  gathered  can  serve  as  excellent  training  data  to  tune 
coevolutionary  agent  models  as  well  as  test  data  against  which  to  validate  the  models. 

While  Corman  represents  the  narrative  in  text,  and  Horgan  represents  the  coevolution  of  conflict  in 
structured  interviews,  Rozenblit  represents  the  coevolution  of  conflict  in  text.  Rozenblit  led 
development  of  ATRAP,  a  toolkit  that  aids  intelligence  analysts  to  find  and  interpret  patterns  in  free 
text  that  could  represent  enemy  COAs.  ATRAP  combines  genetic  algorithms,  game  theory,  and  NLP 
to  give  analysts  an  automated  filter  for  documents  that  together  create  alerts  and  situation 
awareness  of  possible  enemy  actions.  The  system  finds  entities  and  links  in  text  that  form  patterns 
that  an  analyst  can  mark  as  an  indicator.  The  analyst  can  then  build  a  threat  template  by  stringing 
together  indicators  in  parallel  and  in  sequence  to  describe  enemy  tactics,  techniques,  and 
procedures  (TTPs)  or  any  COA. 

In  the  context  of  the  Algerian  example,  the  threat  template  might  string  together  the  three 
indicators  "money  transfer",  "to  organization",  and  "related  to  location  of  interest"  in  parallel  with 
the  three  indicators  "money  transfer",  "to  organization",  and  "related  to  equipment  of  interest." 
The  French  forces  could  have  applied  such  a  threat  template  to  unstructured  text  in  messages, 
such  as  terrorist  interrogation  reports  or  human  intelligence  (HUMINT)  reports  on  hawala 
exchanges,  with  a  location  of  interest  such  as  "Kasbah"  and  equipment  of  interest  such  as 
"explosives."  If  an  ATRAP  template  identified  the  money  transfer  as  a  gold  transaction  over  $1,000, 
it  could  indicate  purchase  of  an  explosive. 

ATRAP's  innovation  lies  in  the  way  templates  model  plausible  changes  in  terrorist  TTPs.  ATRAP  first 
uses  a  genetic  algorithm  to  find  new  patterns  that  resemble  existing  patterns  in  the  text  corpus. 
ATRAP  then  applies  game  theory  to  model  how  this  new  template,  as  a  COA,  would  play  out 
against  existing  defenses  in  a  coevolutionary  dialectic  with  counter-tactics.  Staying  with  the 
Algerian  example,  if  the  insurgents  had  used  game  theory  to  model  the  new  tactic  of  simultaneous 
bombing,  they  might  have  predicted  that  demonstrations  would  occur  if  the  French  employed  a 
kinetic  response  to  counter  a  simultaneous  bombing.  The  French  could  counter  the  demonstrations 
with  an  even  larger  kinetic  response,  which  might  discourage  the  population  from  supporting  the 
insurgency.  To  adapt  their  strategies,  the  insurgents  chose  to  discourage  the  reactive 
demonstrations. 

ATRAP  could  be  further  enhanced  to  include  identification  of  terrorist  and  friendly  narratives  in 
data  and  associated  actions  such  as  CMO  responses  to  insurgent  actions.  To  do  so,  ATRAP  would 
either  have  to  be  augmented  to  find  indicators  of  narratives,  or  work  in  concert  with  other 
programs  that  identify  terrorist  narratives  and  counternarratives,  such  as  Corman's  ITNC.  Once 
modelers  have  formalized  the  representation  of  narratives  and  COA  templates  and  extracted  them 
from  the  text,  they  become  available  for  models  to  ingest  as  "training"  data  or  as  "testing"  data  to 
validate  the  model. 
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4.  Coevolutionary  Integration  of  Models  for  COA  Analysis  of  Social  Institutions 

The  arrangement  and  integration  of  models  is  another  dimension  in  data  processing  for  COA 
analysis.  Model  integration,  because  it  involves  interactions  between  disparate  models  through 
data,  is  a  task  for  data  processing  rather  than  for  modeling.  An  autonomous  agent,  and  the  way  a 
cognitive  autonomous  agent  models  its  environment,  could  be  viewed  as  a  complete  model  in 
itself,  while  the  interface  between  agents  constitutes  a  model  integration  scheme. 

Coevolutionary  integration  of  models  has  particular  importance  in  COA  analysis.  Coevolution  is  a 
model  integration  scheme  that  can  compute  a  population's  reaction  by  integrating  individual 
agents,  each  of  which  uses  an  autonomous  machine  learning  algorithm  for  maximizing  utility— that 
is,  for  best  achieving  goals.  Such  agents  are  autonomous  because  they  calculate  for  themselves 
how  to  perceive  and  act  to  increase  their  utility,  and  do  so  in  a  way  that  makes  sense  in  their 
individual  situation.  Coevolving  agents  determine  how  to  perceive  and  act  by  using  a  method  of 
induction,  such  as  a  neural  network,  genetic  algorithm,  Bayesian  network,  reinforcement  learning 
algorithm,  or  even  a  cognitive  architecture  (Alpaydin,  2009;  Mitchell,  1998;  Sun,  2008).  Inductive 
methods  compute  how  an  agent  should  perceive  and  act  through  trial  and  error  iterations  of 
perceptions  and  actions  based  on  how  these  perceptions  and  actions  affect  utility.  The  important 
aspect  of  model  integration  is  that  the  agents  exert  coevolutionary  pressure  on  each  other, 
meaning  that  they  change  each  other's  utility  values  as  they  act  upon  what  they  learn.  Thus, 
although  agents  may  have  static  overall  goals  such  as  "survival,"  the  most  effective  subgoals  to 
achieve  these  goals  may  change. 

Integrating  individual  agent  "minds"  so  that  they  influence  each  other's  perceptions  is  a  good  way 
to  model  10  COAs  because  the  purpose  of  an  10  model  is  to  explore  why  a  particular  belief  would 
make  sense  to  agents,  whether  human  or  software,  in  their  individual  circumstances.  Models  of  10 
that  do  not  walk  through  sensemaking,  such  as  homophily  models  based  solely  on  copying  agent 
neighbors,  cannot  explain  why  some  10  messages  are  not  copied,  that  is,  why  they  failed  to 
influence  the  intended  audience,  and  therefore  cannot  test  COAs.  Coevolving  agents  are  not  so 
dependent  upon  each  other  as  to  be  empty  of  thought,  like  homophily-based  agents,  nor  are  they 
fully  independent  thinkers  that  have  no  effect  on  each  other:  rather,  they  are  autonomous 
networked  beings,  each  of  whom/which  influences  what  makes  sense  to  the  other. 

The  integration  scheme  between  individual  models  of  thinking  agents  (their  external  interface) 
represents  the  agent's  social  world.  Sociocultural  modeling  focuses  on  what  happens  in  the 
interface:  the  connection  between  the  agents'  minds  (in  other  words,  their  network). 
Understanding  social  behavior  equates  to  understanding  institutions.  Human  institutions  are  social 
because  they  are  based  on  human  agent's  knowledge  of  how  other  human  agents  will  react  to 
their  behaviors.  For  example,  bribery  is  social  because  people  would  not  offer  bribes  if  they  did  not 
have  confidence  that  the  bribe  would  be  accepted,  or  at  least  that  they  would  not  go  to  jail 
because  of  the  bribe.  To  achieve  their  goals,  agents  learn  corresponding  behaviors,  such  as  to  bribe 
and  to  accept  a  bribe.  As  these  behaviors  become  expected,  they  change  the  utility  of  the  agent 
behaviors,  so  that  bribing  increases  utility  only  if  the  bribes  are  accepted.  As  these  behaviors 
become  commonplace,  they  become  institutions. 
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The  Battle  of  Algiers  scenario  shows  the  alignment  of  goals  into  acceptable  behaviors  that  rely 
upon  established  institutions,  such  as  rules  about  actions  that  merit  support.  For  example,  the 
strike  that  the  FLN  initiated  was  not  large  enough  to  garner  support  from  the  UN,  but  the  French 
suppression  of  the  strike  was  violent  enough  to  be  distasteful  to  the  European  French  and  reduce 
their  support  of  the  war  effort. 

The  interdependence  of  behaviors,  to  include  interlocking  and  corresponding  behaviors,  causes 
societies  to  develop  into  distinct  types  in  which  not  every  possible  combination  is  possible. 
Behavioral  dependency  makes  social  behaviors  self-reinforcing,  causing  virtuous  or  vicious  cycles. 
COA  analysis  of  an  entire  society,  therefore,  amounts  to  the  study  of  when  society  tips  from  one 
self-reinforcing  set  of  behaviors  to  another,  and  10  COA  analysis  becomes  the  study  of  why  a 
paradigm  shifts  from  one  way  of  viewing  the  world  to  another.  That  paradigm  shift  in  viewpoint  is 
critical  to  the  behavioral  change. 

The  Nexus  Cognitive  Agent  Program  (Nexus),  created  at  the  Office  of  the  Secretary  of  Defense 
(OSD),  modeled  corruption  and  popular  support  in  OSD's  first  IW  Analytical  Baseline,  the  Africa 
Study  (Duong,  2012).  In  Nexus,  corruption  is  modeled  as  a  system  of  interlocking  behaviors  that 
agents  learn  to  execute  (or  not)  depending  on  how  the  behaviors  affect  utility.  Utility  in  Nexus  is 
defined  by  cultural  values.  For  example,  in  the  Battle  of  Algiers  scenario,  both  the  general 
population  and  the  insurgents  had  cultural  goals  such  as  eliminating  prostitution  and  alcohol 
consumption.  In  the  Africa  Study  corruption  scenario,  software  agents  represented  human 
individuals  and  utility  was  defined  by  African  tribal  cultural  values.  For  example,  a  tribesman  from  a 
matrilocal  tribe  may  care  that  his  maternal  grandmother  has  bought  enough  food  in  the  market  in 
the  past  year  to  sustain  her  health.  Choices  to  give  and  receive  bribes  only  indirectly  affect  utility, 
but  indirectness  is  important  because  we  can  effectively  explore  whether  bribery  is  effective  or  not 
under  a  variety  of  circumstances.  If  our  simulation  could  produce  no  other  outcome— that  is,  if 
bribing  always  generated  positive  outcomes  and  the  optimizing  agent  has  no  recourse  but  to 
bribe— then  we  cannot  report  to  senior  leadership  that  we  have  studied  bribing  with  scientific 
validity. 

In  the  Africa  Study  corruption  scenario,  Nexus  agents  possessed  a  strategy  of  behaviors,  telling 
them  to  bribe,  accept  bribes,  steal,  or  tolerate  stealing,  along  with  a  variety  of  network  choice 
behaviors.  Each  strategy  was  a  chromosome  in  a  Bayesian  Optimization  Algorithm  (BOA).  Each 
agent  had  its  own  private  BOA  with  20  strategies,  defined  as  a  combination  of  behaviors,  and 
executed  each  strategy  for  one  year.  The  strategies  were  ranked  by  utility  of  the  agents' 
dependents,  for  example,  the  maternal  relatives  of  matrilocal  agents.  The  10  best  strategies  were 
retained  and  Nexus  then  created  ten  more  new  strategies  with  the  same  statistical  properties  as 
the  ten  best. 

One  particular  analysis  that  can  be  conducted  utilizing  the  Nexus  tools  involves  testing  the  effect  of 
transparency  programs  versus  higher  judicial  penalties  on  the  society  to  determine  which  COAs  will 
tip  the  society  from  corrupt  to  noncorrupt  states.  For  example,  analysts  can  test  for  a  reduction  of 
bribery  transactions  between  agents,  such  that  agents  no  longer  expect  to  receive  bribes  from 
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other  agents  or  feel  obligated  to  offer  them,  or  examine  other  goals  that  the  COAs  attempt  to 
affect. 

In  the  Nexus  Africa  Study's  popular  support  scenario  (Marling,  Duong,  Sheldon,  Stephens,  Murphy, 
Johnson,  &  Ottenberg,  2008),  every  agent  maintained  a  neural  network  with  which  to  induce 
popular  support  and  determine  whom  to  credit  and  whom  to  blame  for  actions  based  on 
consistency  theories  in  social  psychology.  The  model  implemented  Festinger's  cognitive  dissonance 
theory  (Festinger,  1957),  Heider's  balance  theory  (Heider,  1946),  and  Fischer's  narrative  paradigm 
(Fischer,  1984).  Nexus  assigns  each  agent  a  constraint  satisfaction  neural  network,  specifically,  a 
Boltzmann  machine,  to  represent  the  mind.  In  this  network  other  agents  tend  to  be  supported  if 
they  treat  the  agent  well  and  support  the  agent's  "friends,"  and  tend  not  to  be  supported  if  they 
treat  the  agent  poorly  or  do  not  support  the  agent's  friends.  Historical  actions  and  a  network  of 
support  all  contribute  to  an  agent  "schema"  of  support.  That  schema  consists  of  a  set  of  reinforcing 
facts  that  cause  an  agent  to  polarize  friends  and  enemies,  and  perhaps  to  blame  its  enemies  for 
more  than  they  have  done  while  perhaps  supporting  its  friends  more  than  they  deserve.  Support 
relations,  such  as  coalitions,  that  link  the  agents  arise  from  these  schemata  in  the  form  of  public 
declarations  of  support;  because  they  are  self-reinforcing  they  tend  to  tip  the  society  between 
states.  The  schema  enables  COA  analysis  of  popular  support,  such  as  how  coalitions  change  as  a 
result  of  exogenous  interventions.  The  coevolutionary  integration  scheme  between  autonomous 
agent  models  is  critical  to  the  COA  analysis,  because  popular  support  takes  the  form  of  data  traded 
between  the  models,  and  coevolution  of  this  data  represents  how  individual  agents  change  each 
other's  minds. 


5.  Game  Theoretical  Integration  of  Models  for  COA  Analysis  of  Strategic  Moves 
and  Countermoves 

Recall  that  not  only  software  agents  but  also  their  cognitions  about  their  environment  are 
individual  models.  Models  can  represent  a  strategist  agent's  situation  awareness,  and  can  be 
executed  in  a  way  that  represents  each  side's  mental  model  of  the  enemy  and  of  the  host 
population.  Thus  models  can  be  integrated  in  a  game-theoretical  manner  for  analysis  of  strategic 
moves  and  countermoves.  In  this  way  they  can  represent  a  mental  model,  or  "what  if  he  thinks  X..." 
scenario,  even  if  they  were  designed  as  models  of  the  environment.  For  example,  the  extensible 
Behavioral  Model  integration  framework  (XBM)  project  led  by  Impact  Computing  (2013)  is  an  open 
source  tool  that  enables  the  representation  of  a  strategist  agent's  situation  awareness  by  running 
any  models  (not  just  cognitive  models)  as  though  they  were  the  strategists'  mental  models. 

XBM  runs  models  in  innovative  strategic  combinations  with  each  other  for  COA  analysis. 
Specifically,  it  supports  the  concept  of  strategic  data  farming,  a  variation  of  data  farming  that 
utilizes  game  trees  to  detect  best  COAs  during  scenario  runs  (Duong,  Brown,  Schubert,  McDonald, 
Makovoz,  &  Singer,  2010).  Data  farming  is  a  form  of  COA  analysis  by  testing  parameter  sensitivity 
through  a  very  large  number  of  model  execution  runs.  One  way  to  test  COAs  with  computer 
simulations  is  to  trace  which  inputs,  such  as  IW  actions,  lead  to  which  outputs,  such  as  popular 
support.  Again  using  the  Battle  of  Algiers  scenario,  inputs  for  the  French  commanders  could  be 
kinetic  strategies  to  eliminate  the  insurgents  through  checkpoints,  curfews,  and  harsh  treatment, 
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while  outputs  of  interest  would  include  estimates  of  popular  support  among  the  indigenous 
Algerians  and  the  European  French.  Data  farming  is  complicated  because  the  large  number  of 
parameters  in  sociocultural  simulations  typically  precludes  testing  every  parameter  in  combination 
with  all  the  others  to  see  what  combinations  of  inputs  lead  to  what  outputs.  Testing  parameters  in 
combination  could  require  billions  and  billions  of  simulation  runs;  a  stochastic  simulation  would 
demand  an  order  of  magnitude  more  runs  to  cover  the  space  of  possibilities. 

Data  farming  covers  the  parameters  as  evenly  as  possible  by  applying  methods  such  as  Cioppa's 
"orthogonal  Latin  hypercubes,"  which  optimize  parameter  coverage  on  a  simulation  treated  as  a 
black  box  (Cioppa  &  Lucas,  2007).  Strategic  data  farming  has  a  different  focus:  rather  than 
examining  all  possible  outputs  from  the  simulation,  it  performs  risk-based  COA  analysis  by  sampling 
the  output  space  in  proportion  to  likelihood.  For  example,  in  the  Algerian  scenario,  kinetic  input 
moves  of  the  French  government  would  usually  correspond  to  input  moves  of  the  insurgents  such 
as  strikes.  Because  simulation  parameters  are  not  in  fact  orthogonal,  strategic  data  farming  inputs 
parameters  proportionately  to  generate  a  proportional  output  space. 

Using  strategic  data  farming,  XBM  can  also  represent  deception,  a  concept  central  to  10  campaigns. 
In  XBM,  an  analyst  may  execute  a  model  scenario  depicting  the  red  side's  concept  of  the  blue  side's 
concept  of  the  battlefield.  One  example  of  deception  is  the  IW  tactic  of  trying  to  incite  trouble  so 
that  the  other  side  seems  brutal  and  receives  the  blame  of  the  population.  This  tactic  appeared  in 
the  Algiers  example  when  the  insurgents  bombed  French  citizens,  eliciting  a  harsh  reaction  from 
the  French  police.  XBM  could  execute  a  model  that  represents  the  indigenous  Algerian  population's 
reaction  to  French  kinetic  and  nonkinetic  moves  as  the  insurgents'  cognition  regarding  the 
consequences  of  their  own  actions  meant  to  incite  the  French.  Executing  these  coexisting  situation 
awarenesses  and  resultant  behaviors  results  in  a  COA  analysis  based  on  strategic  moves  and 
countermoves. 

Analysts  at  the  U.S.  Army  Training  and  Doctrine  Command  Analysis  Center  (TRAC)  applied  strategic 
data  farming  to  an  IW  10  scenario  using  XBM  and  Nexus.  The  research  evaluated  moves, 
countermoves,  and  strategies  from  TRAC's  tactical  wargame  to  determine  the  best  COAs  under 
certain  conditions.  The  research  also  provided  insight  into  the  population's  attitude  toward  the 
host  nation  and  insurgents,  and  helped  identify  tipping  points  where  the  population  may  shift 
allegiance. 

XBM  performs  dynamic  moves  to  achieve  agent  objectives  by  implementing  the  following  three 
capabilities:  (1)  assessing  indicators  that  measure  how  far  agents  are  from  their  goals  at  specific 
time  intervals,  (2)  saving  and  precisely  restoring  the  simulation  state  (checkpoint/restart),  and  (3) 
implementing  moves.  XBM  defines  player  strategies,  and  players'  perceptions  of  other  players' 
strategies,  in  advance.  In  strategic  data  farming,  a  strategy  consists  of  the  following  elements  from 
the  Military  Decision  Making  Process  (MDMP): 

•  Decision  points  -  conditions  under  which  a  change  of  Concept  of  Operations  (CONOPS)  is 
to  be  considered 

•  Branches  and  sequels  -  doctrinal  guidelines  that  constrain  and  define  possible  moves 
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•  Goal  states  -  ultimate  objectives 

•  Subgoal  states  -  proximate  short-term  objectives 


XBM  maintains  "ground  truth"  that  represents  reality  as  opposed  to  any  particular  player's  view  of 
reality.  When  a  player's  decision  point  has  triggered  and  the  player  must  choose  among  doctrinal 
moves,  the  simulation  is  executed  from  the  viewpoint  of  the  decider  (decision  maker).  In  other 
words,  the  simulation  executes  the  decider's  idea  of  the  other  players'  points  of  view  as  the 
decider  looks  ahead  (mental  model)  to  what  might  happen.  If  perception  and  deception  are 
excluded  only  one  model  is  needed,  because  blue's  view  of  red's  plans  and  what  red  sees  are 
assumed  to  be  correct  and  vice  versa. 


When  the  decider  reaches  a  decision  point,  or  believes  that  other  players  have  reached  a  decision 
point,  the  simulation  branches  (along  the  game  tree)  according  to  the  top  n  moves  of  that  player's 
strategy.  Other  player's  moves  are  translated  back  to  the  decider's  simulation,  which  represents 
the  decider's  ground  truth.  This  pattern  recurs  until  a  certain  number  of  decision  points  have  been 
reached,  and  then  XBM  judges  each  separate  simulation  according  to  the  decider's  idea  of  the 
other  player's  goals.  XBM  sends  the  score  of  the  simulation  back  up  the  mental  model  game  tree, 
assuming  each  player  selects  actions  that  lead  to  the  most  favored  condition.  The  decider  then 
picks  its  next  move  with  the  assumption  that  each  player  is  optimizing  its  goal  at  every  decision 
point.  All  players  decide  their  next  moves  in  the  same  manner,  and  then  the  ground  truth 
simulation  advances  to  the  next  decision  point. 


Figure  1  illustrates  a  small  portion  of  a  game  tree  in  which  blue  looks  ahead  for  six  decision  points. 
Ratings  are  shown  for  each  simulation  state  from  the  bottom  of  the  tree  (not  shown). 


•  Actual  Simulation  Run. 


o 


Figure  1.  One  decision  point  is  reached  in  a  simulation,  causing  blue  to  examine  six  decision  points 
and  choose  a  COA  according  to  its  CONOPS. 


372  |  Data  interfaces  for  input,  output ,  and  translation  between  models 


Figure  2  illustrates  an  example  run  of  the  Nexus  model.  In  the  Africa  Study,  Nexus  modeled  and 
measured  intergroup  support  levels.  Nexus  represented  a  "group  mind"  for  each  tribe  and 
computed  support  levels  based  on  past  historical  events  and  support  networks.  The  study 
measures  intergroup/alliance  strength  by  executing  stochastic  Nexus  ten  times.  If  a  group  mind 
was  positive  about  another  group  in  every  case,  then  the  alliance  was  deemed  solid.  If  a  group 
mind  was  positive  about  another  group  half  the  time  and  negative  half  the  time  the  alliance  was 
deemed  tippable. 


Start 


Green  Action 


Red 

Reaction 


RE:  .35  RE:  .65  RE:  .75 


RE:  .5 


Green  Goals 

•  Separate  Tribe  J  from  the 
insurgency. 

•  Improve  Tribe  J  sentiment  toward 
green. 

Red  Goals 

•  Improve  Tribe  J  sentiment  toward 
insurgency. 

•  Separate  Tribe  J  from  green. 


Green  Move 

1 .  Disrupt  alliance  between  tribe  J 
and  tribe  D. 

2.  Conduct  Civil  Affairs. 

Red. Move 

3.  Make  tribe  O,  a  green  ally,  appear 
to  harm  tribe  J. 

4.  Make  green  appear  to  harm  tribe 
J. 


Figure  2.  Goals,  doctrinal  moves,  and  evaluation  function  (GE  and  RE)  of  a  Nexus  strategic  data 
farming  walkthrough. 


In  this  scenario,  green  represents  the  host  nation  government  actions  and  red  represents  insurgent 
actions.  Both  green  and  red  focused  their  attention  on  the  opinion  of  Tribe  J,  the  main  tribe  from 
which  the  insurgency  recruited  its  members.  All  the  people  of  Tribe  J  initially  had  positive  views  of 
the  insurgents,  even  though  57%  of  Tribe  J  also  had  positive  attitudes  towards  the  green 
government. 

Green  wanted  to  act  in  the  region  because  its  popular  support  had  waned.  The  objective  function 
was  to  form  a  more  solid  alliance  with  one's  group  and  a  more  solid  opposition  with  the  opponent. 

Green's  evaluation  function  was:  GE=  ((the  ratio  of  times  green's  support  node  was  illuminated  in 
Tribe  J's  brain)  +  (the  ratio  of  times  red's  support  node  was  illuminated  in  Tribe  J's  brain))/2.  Red's 
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evaluation  function  was:  RE  =  1-  GE.  An  increase  or  decrease  in  the  evaluation  function  served  as 
the  decision  point.  Fully  executing  the  single  mind  of  Tribe  J  in  Nexus  with  no  moves,  to  get  a 
baseline  of  what  would  happen  in  the  simulation  had  no  party  intervened,  resulted  in  a  tight 
alliance  between  Tribes  J  and  D,  and  a  GE  score  of  about  .28. 

Assume  green's  strategy  involves  a  choice  of  either  performing  positive  civil  affairs  actions  or  trying 
to  disrupt  the  alliance  between  Tribes  J  and  D  by  causing  ethnic  tension  (for  example,  having  a 
Tribe  J  member  provoke  an  attack  by  Tribe  D  members,  filming  it,  and  broadcasting  it  to  Tribe  J 
members).  When  green  assesses  these  moves  in  isolation,  both  increase  GE  to  0.5.  To  differentiate 
between  the  two,  green  must  look  at  the  vulnerabilities  its  actions  may  have  created  for  red 
retaliation.  Say  that  the  red  strategy  only  includes  making  others  look  bad  (for  example,  by 
provoking  green  into  acts  of  violence  against  Tribe  J,  and  by  inciting  ethnic  tensions  between  Tribe 
J  and  a  green  ally,  Tribe  0).  If  green  only  performs  civil  affair  actions,  inciting  tension  between  Tribe 
J  and  Tribe  0  causes  RE  to  reach  0.75.  Green  acts  of  violence  against  Tribe  J  result  in  an  RE  score  of 
0.5.  Thus,  since  red  would  choose  to  incite  tension  (because  it  would  cause  more  damage),  the  true 
value  of  the  civil  affairs  action  to  green  is  GE  =  1  -  0.75  =  0.25. 

On  the  other  hand,  if  green  first  causes  ethnic  tension  with  Tribe  D  and  red  retaliates  by  causing 
ethnic  tension  with  Tribe  O,  then  RE  =  0.35,  and  if  red  retaliates  by  inciting  acts  of  violence  by 
green  against  Tribe  J,  RE  =  0.65.  Thus,  in  this  case,  red  would  again  choose  to  incite  acts  of  violence 
against  Tribe  J,  the  action  with  the  strongest  effect.  Apparently,  if  green  incites  ethnic  tensions 
first,  it  is  protected  from  the  effects  of  red  actions  later.  Accordingly,  the  value  of  inciting  ethnic 
tensions  to  green  is  GE  =  1  -  0.35  =  0.65.  Since  0.65  is  greater  than  the  present  value  of  0.28,  it  is  to 
green's  advantage  to  take  this  action,  given  the  strategy  and  goals  of  both  sides. 

Another  model  that  integrates  agent  minds  that  interact  and  think  through  consequences  of  their 
behavior  is  Senturion,  a  proprietary  model  of  Sentia,  Inc.  (Kugler  &  Abdollahian,  2013).  Senturion 
simulates  the  political  bargaining  process  on  the  basis  of  two  economic  theories:  median  voter 
theory  and  expected  utility  (EU)  theory.  Median  voter  theory  is  a  technique  for  predicting  the 
stance  that  stakeholders  (for  example,  a  political  candidate)  might  take  on  an  issue  in  order  to  win 
the  most  support.  Senturion  uses  game  theory  and  the  assumption  that  the  candidate  sells  out  his 
actual  position  in  order  to  win  votes,  where  the  winning  position  is  the  median. 

Senturion  uses  EU  theory  to  incorporate  a  form  of  perception  and  cognitive  bias  -  risk  aversion  - 
into  this  purely  rational  game  theory  technique.  Instead  of  assuming  that  the  politician  will  do 
anything  to  win,  Senturion  calls  the  selling  out  of  one's  beliefs  "risk  aversion"  and  represents  the 
tradeoff  between  that  and  retaining  one's  beliefs  as  "risk  acceptance."  Senturion  identifies  the 
stakeholders  near  the  median  issue  stance  as  risk  averse  and  likely  to  give  in,  while  stakeholders  far 
from  the  median  accept  risk  and  are  unlikely  to  move  far  from  their  positions. 

Senturion  uses  parameters  that  include  the  salience  of  an  issue  to  a  stakeholder,  and  stakeholder 
influence,  to  calculate  a  "weighted  median."  In  each  iteration  of  Senturion,  agents  make  proposals 
to  all  other  agents  regarding  position  change,  and  every  agent  then  chooses  one  proposal  and 
moves  toward  it.  In  deciding  what  proposal  to  accept  agents  consider  how  much  influence  a 
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stakeholder  wields  and  how  important  the  issue  is  to  the  stakeholder.  Additionally,  a  risk-averse 
decider  magnifies  how  much  power  another  stakeholder  has,  while  a  risk-acceptant  decider 
disregards  another  stakeholder's  power.  Agents  assess  where  they  would  land  if  they  took  an  offer, 
and  then  choose  the  offer  that  causes  them  to  move  as  little  as  possible  from  their  present 
position. 

EU  is  well  vetted  and  accepted  by  the  modeling  community.  It  combines  the  two  most  important 
components  of  decision  making:  rationality  and  bias.  Thus,  EU  captures  the  tradeoff  between  the 
idealist's  adherence  to  principle  and  the  practical  bow  to  social  pressure  that  permeates  politics. 

Senturion  is  the  best  current  model  of  political  bargaining,  and  is  highly  useful  for  the  study  of  IW. 
In  the  Africa  Study,  Senturion  represented  popular  support  of  the  host  nation  as  the  issue  about 
which  the  stakeholders  should  arrive  at  consensus.  In  the  Battle  of  Algiers  scenario  this  would  be 
the  equivalent  of  modeling  the  insurgents,  the  Algerian  indigenous  population,  the  French 
Algerians,  and  the  European  French  as  stakeholders,  inserting  kinetic  and  nonkinetic  COAs  for  each, 
and  computing  their  compromises  based  on  their  relative  power. 


6.  Preserving  Uncertainty  for  Risk-based  COA  Analysis 

New  data  processing  methodologies  for  COA  analysis  include  representations  of  narratives,  COA 
templates,  and  elements  of  the  MDMP.  They  also  include  methods  for  integrating  models  to 
compute  social  network  behaviors  to  test  interventions,  as  well  as  the  strategic  interactions  of  IW. 
However,  one  especially  important  methodology  that  research  must  advance  is  tracking 
uncertainty  for  risk-based  COA  analysis.  In  conventional  warfare  analysis,  uncertainty  stems  from  a 
few  random  variates  from  distributions  inside  physics-based  models.  IW  analysis  must  take  into 
account  uncertainty  in  data,  uncertainty  on  the  correctness  of  social  theory,  uncertainty  on  the 
correctness  of  the  representation,  uncertainty  in  the  way  to  translate  data  between  disparate 
models  and  data  created  for  different  purposes,  and  even  intrinsic  uncertainties  of  human 
behavior. 

Innovative  methodologies  help  track  these  uncertainties.  For  example,  researchers  could  mitigate 
the  uncertainty  about  social  theory  by  testing  the  theories  underlying  different  models  in 
proportion  to  their  reliability  and  testing  strategies  for  robustness  against  all  theories.  Because 
there  are  so  many  possibilities,  sociocultural  analysis  requires  multiple  models  that  depend  on 
integration  frameworks  to  track  the  uncertainty.  Integration  frameworks  such  as  Oz,  Global 
Information  Network  Architecture  (GINA),  XBM,  and  SIMmiddleware  translate  between  models 
using  ontologies  in  hub-and-spoke  arrangements.  An  ontology  is  a  grouping  of  data  into  the 
language  of  a  domain,  along  with  the  rules  for  grouping  the  data.  Ontologies  are  used  for 
translation  because  they  define  their  terms  precisely.  A  probabilistic  ontology  additionally 
represents  uncertainty  in  knowledge  about  how  data  are  grouped  and  how  they  should  be 
translated  by  combining  crisp  ontological  logic  with  Bayesian  inference. 

For  the  TRAC  tactical  wargame,  SIMmiddleware  integrated  hub,  spoke,  and  mediation  probabilistic 
ontologies  to  support  analysis  (Duong  &  Bladon,  2012).  These  ontologies  tracked  the  uncertainty  of 
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data  translation  so  that  the  results  kept  track  of  the  likelihood  of  a  particular  outcome.  The 
uncertainties  within  the  ontologies  were  combined  using  rule-based  systems.  For  example,  the 
translation  between  the  Cultural  Geography  (CG)  model  (one  of  the  tactical  wargame  models)  and 
the  hub  model  required  probabilistic  translation.  Specifically,  analysts  created  a  one-to-many 
mapping  of  CG  moves  to  hub  model  moves  that  SIMmiddleware  implemented  using  probabilistic 
ontologies.  Probabilistic  ontologies  were  also  used  to  assess  crisp  rules  to  fire  decision  points  based 
on  popular  support  indicators.  Probabilistic  ontologies  such  as  Bayes  Owl  combine  probabilistic  and 
crisp  logic  (Ding,  Peng,  &  Pan,  2004). 

XBM  provides  a  way  to  take  data  credibility  into  account  in  a  proportionate  output  using  a  mixing 
function.  A  mixing  function  draws  proportionately  from  input  data  sets  according  to  their  reliability, 
using  XBM's  probabilistic  ontology,  ProbOnt.  SIMmiddleware  provides  a  way  to  track  these  forms 
of  uncertainty,  data  pedigree,  social  models,  and  model  translation,  so  as  to  output  the  results  of 
multiple  runs  proportionately  into  a  Markov  process.  A  Markov  process,  a  node-and-link  graphical 
model  that  displays  the  probability  of  going  from  one  state  to  another,  describes  the  state  space 
(all  possible  states)  probabilistically.  The  Markov  process  can  be  used  to  analyze  likely  tipping 
points.  For  example,  to  measure  the  effectiveness  of  an  action,  the  process  can  calculate  the 
likelihood  of  reaching  any  state  from  the  point  of  the  action  and  can  show  whether  selected 
actions  can  achieve  desired  goal  states.  The  process  can  also  analyze  actions  that  drive  the  system 
to  states  in  which  the  goal  is  more  easily  attainable. 


7.  Data  Processing  Implications  of  Validation  of  COA  Analyses 

Although  IW  scenarios  in  simulation  models  are  individualistic,  research  can  and  should  extract 
generalities  from  these  scenarios.  For  example,  analysts  can  measure  whether  the  correlation  of 
states  in  the  model  mirrors  their  correlation  in  the  real  world,  for  the  purpose  of  validation. 
Generalization  is  most  fundamentally  an  issue  of  data  representation,  and  finding  the  right  level  of 
generalization  in  an  ontology  that  best  establishes  a  correlation  is  a  data  processing  issue. 

TRAC  used  SIMmiddleware  to  study  the  validity  of  models  of  its  2010  tactical  wargame  by 
comparing  model  output  to  Afghan  Nationwide  Quarterly  Assessment  Review  (ANQAR)  data  from 
the  same  year  portrayed  in  the  tactical  wargame.  Analysts  calculated  a  probabilistic  distance 
between  the  model  and  the  real-world  data  using  the  Kullback-Leibler  convergence  probabilistic 
distance  function.  This  equation  represents  the  information  lost  when  one  Markov  process  is 
substituted  for  another;  specifically,  the  number  of  bits  needed  to  make  the  output  of  one  match 
the  other.  The  Markov  process  shown  in  Figure  3  illustrates  results  of  the  validity  research  from 
tactical  wargame  2010.  Nodes  indicate  types  of  player  actions  and  the  popularity  of  a  coalition 
player.  Arcs  represent  the  likelihood  of  transitioning  between  node  states.  Green  areas  represent 
'good'  states  (popular)  while  red  areas  represent  'bad'  states  (unpopular).  The  probabilistic 
distance  between  the  model  and  the  real-world  data  shows  a  score  of  0.21,  a  relatively  low 
probabilistic  distance  (lower  is  better).  This  score  served  as  evidence  for  the  validation  of  the 
tactical  wargame  model  at  a  rudimentary  level.  Since  SIMmiddleware  and  its  implemented 
probabilistic  ontologies  preserved  the  uncertainties  in  the  data,  models,  and  translations  between 
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models,  the  Markov  process  represents  the  best  estimate  of  the  probabilities  of  arriving  at 
different  states  of  interest  (Duong  &  Bladon,  2012). 


State  Definition 

Level  of  Violence 

Popularity  of  Green 

K  =  Kinetic 

P  =  Popular 

M  =  Medium  Kinetic 

U  =  Unpopular 

N  =  Non-Kinetic 

Model  State  Transitions 


Level  of  Violence 

Medium  Kinetic 


Level  of  Violence 

Medium  Kinetic 


Start  Scenario 


•  Apply  Kullback-Leibler  (KL)  divergence  to  measure  probabilistic  distance  between 
Markov  processes:  dkl(P||Q)  =  £p(i)logg|. 

•  Score  of  0  means  exact  same  Markov  process.  Score  of  1  means  the  most  different 
Markov  process  possible.  Example  above  generated  KL  score  =  0.21. 


Figure  3.  Comparison  of  Markov  processes  for  validation  score. 


The  concepts  of  a  social  theory  should  enable  output  data  from  IW  models  to  correspond  to  real- 
world  data.  The  validation  analysis  from  Figure  3  compared  model  and  real-world  data  regarding 
simple  concepts  such  as  popularity  and  kinetic  actions.  However,  IW  concepts  are  much  more 
complex,  and  models  representing  IW  concepts  must  be  formalized  to  determine  whether  they 
align  with  theory. 

The  U.S.  Army  Engineer  Research  and  Development  Center's  (ERDC)  Cultural  Reasoning  and 
Analysis  for  the  Tactical  Environment  (CREATE)  offers  a  good  example  of  such  a  formalization  that 
could  be  used  to  objectively  score  how  closely  a  simulation  conforms  to  theory  (Whalley,  Perkins, 
Krooks,  Hargrave,  &  Rewerts,  2012).  CREATE  is  unique  in  its  formalization  of  the  arguments  from 
social  literature  that  frame  the  situation  in  a  host  country.  CREATE's  factor  map  formalizes  the 
social  literature  on  CMO  with  an  augmented  version  of  Shum's  Naturalistic  Argumentation  upper 
ontology  (Shum,  Uren,  Li,  Sereno,  &  Mancini,  2007).  Shum's  ontology  consists  of  relations  between 
entities  that  indicate  the  degree  to  which  social  literature  claims  support  or  refute  one  another. 
The  social  literature  is  annotated  with  these  relationships,  so  that  the  analyst  can  easily  query 
scenario  data  that  support  the  social  model/frame/theory.  For  example,  a  theory  may  claim  that 
insurgency  is  political  in  nature  while  another  theory  may  identify  poverty  as  the  root  cause  of 
insurgency. 
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Using  CREATE's  annotation  user  interface,  an  analyst  can  manually  mark  claims  in  the  text  as  major 
claims  of  the  article,  and  then  mark  all  supporting  claims  with  links  labeled  "is  evidence  for," 
"proves,"  etc.  (see  Table  1).  The  claims  and  the  link  between  them  form  an  rdf  triplet  for  a  Web 
Ontology  Language  (OWL)  ontology.  The  links  are  weighted  to  show  the  level  of  support  (or 
refutation)  one  claim  has  for  another.  These  claims  are  then  linked  to  claims  in  other  articles  as 
well,  to  include  link  labels  such  as  "is  inconsistent  with."  The  claims  may  represent  any  level  of 
generality,  from  generalizations  about  insurgency  to  specific  facts  about  the  insurgency  in  Kenya. 
Use  of  CREATE  results  in  a  summarized  understanding  of  the  situation  in  terms  of  the  processes 
that  belong  to  a  frame  (for  example,  whether  the  insurgency  in  Kenya  is  caused  by  poverty,  and  if 
so,  the  effectiveness  of  the  intended  CMO  interventions). 

Table  1.  Shum's  Naturalistic  Argumentation  Upper  Ontology 


Relation  Class 

Dialect  label 

Polarity/  Weight 

General 

is  about 

-VI 

uses/applies/is  enabled 
by 

-Vl 

improves  on 

-V2 

impairs 

—  /2 

other  link 

-Vl 

Problem  Related 

addresses 

-Vl 

solves 

-V2 

Supports'"  Challenges 

proves 

-V2 

refutes 

-?2 

is  evidence  for 

-Vl 

is  evidence  against 

-vi 

agrees  with 

-Vl 

disagrees  with 

^1 

is  consistent  with 

-Vl 

is  inconsistent  with 

-/I 

Causal 

predicts 

-Vl 

envisages 

-Vl 

causes 

-V2 

is  capable  of  causing 

-VI 

is  prerequisite  for 

-Vl 

prevents 

—i2 

is  unlikely  to  a  Sect 

-Vl 

Similarity 

is  identical  to 

-V2 

is  similar  to 

-Vl 

is  different  to 

-1 

is  the  opposite  of 

-V2 

shares  issues  with 

-Vl 

has  nothing  to  do  with 

-Vl 

is  analogous  to 

-Vl 

is  not  analogous  to 

-VI 

Taxonomic 

part  of 

-VI 

example  of 

-VI 

subclass  of 

-VI 

not  part  of 

-VI 

not  example  of 

-1 

not  subclass  of 

-n 

Note:  Table  adapted  with  permission  from  Shum,  Uren,  Li,  Sereno,  &  Mancini  (2007) 
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Applying  this  method  to  an  example  from  the  Battle  of  Algiers  scenario,  we  might  find  that  a  news 
article  states  that  many  of  the  insurgents  are  well  educated.  That  would  conflict  with  the  social 
theory  that  poverty  leads  to  insurgency,  stated  in  another  theoretical  journal  article.  In  this  case, 
the  triplet  for  the  news  article  may  be  <Ahmed,  education  level,  PhD>  and  the  journal  article  triplet 
insurgents,  economic  class,  poverty>.  The  Shum  ontology  triplet  would  be  «Ahmed,  education 
level,  PhD>,  is  evidence  against, insurgents,  economic  class,  poverty».  When  this  connection  is 
found,  the  social  theory  that  poverty  leads  to  insurgency  would  receive  a  “-V .  Even  an  analyst  who 
lacks  a  formal  background  in  the  social  sciences  can  rapidly  assess  the  arguments  and  evidence  that 
explain  the  social  situation  and  comprehend  the  advantages  and  disadvantages  of  varying 
perspectives. 

Formalizing  the  claims  of  social  science  and  applying  them  to  particular  scenarios  of  IW  aids  in 
model  validation.  The  numerical  degree  of  refutation  or  support  of  a  theory  encoded  in  the  Shum 
ontology  links  can  form  an  objective  measure  of  how  much  the  model  output  matches  particular 
social  theories.  Methods  that  enable  objective  measurement  of  models  significantly  advance 
sociocultural  model  validation  requirements. 

Although  the  CREATE  methodology  was  designed  for  analysts  rather  than  computer  models,  its 
formalization  of  the  ideas  of  sensemaking  can  model  the  insights  that  MISO  makes  regarding  the 
host  population,  which  in  turn  are  instantiated  in  CMO  interventions.  The  usefulness  of  CREATE  for 
modeling  CMO  resides  in  its  potential  for  modeling  effects  of  a  MISO  argument  on  a  host  nation 
population  and  to  counter  a  terrorist  narrative.  A  major  gap  in  the  ability  to  model  the  effect  of 
CMO  on  the  host  population  is  the  ability  to  model  MISO.  Doctrine  states  that  CMO  is  the  evidence 
of  MISO,  and  that  the  purpose  of  CMO  is  ultimately  to  influence  popular  support.  However,  almost 
all  models  of  popular  support  are  based  on  message  dissemination  dynamics  by  affinity  rather  than 
on  the  content  of  the  message  itself  and  why  that  message  would  resonate  with  the  people. 

To  model  MISO,  we  must  therefore  model  why  a  MISO  message  will  resonate  with  the  population 
and  how  CMO  supports  the  message.  In  terms  of  natural  argumentation,  the  host  nation 
population  would  have  several  ways  of  framing  the  situation:  in  terms  of  their  cultural  narratives, 
the  CMO  actions  and  the  MISO  that  depicts  them,  and  past  perspectives.  Insurgents  would  give 
evidence  that  supports  some  of  the  population's  frames  and  refutes  others.  Introducing  message 
content  into  the  modeling  of  MISO  would  be  a  vast  improvement  over  existing  algorithms. 


8.  Model  Process  Control 

Model  process  control  (MPC)  uses  feedback  between  data  and  models  to  improve  both  the 
interpretation  of  the  data  and  the  accuracy  of  the  model.  We  have  described  methodologies  to 
extract  data,  input  and  execute  data  in  models,  and  compare  model  output  to  real-world  data.  One 
data  processing  methodology  for  COA  analysis  involves  putting  all  of  these  activities  together  into  a 
feedback  loop  that  interprets  data  and  refocuses  models  on  more  salient  data.  A  feedback  loop 
benefits  all  stages  of  data  fusion  because  even  such  fundamental  steps  as  the  parsing  of  free  text 
data  depend  on  meaning.  For  example,  the  difference  in  the  parse  of  the  statement  "Jack  ate  the 
salad  with  chopsticks"  and  the  statement  "Jack  ate  the  salad  with  croutons"  results  entirely  from 
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the  meanings  of  words.  Applied  to  the  Battle  of  Algiers  scenario,  a  text  may  mention  a  man  named 
Ahmed  who  bought  fertilizer— a  substance  that  may  be  used  to  nourish  crops  or  to  make  bombs. 
The  first  interpretation  would  be  the  triplet  <Ahmed,  purchase,  bomb-materials>  and  the  second 
<Ahmed,  purchase,  farming-materials>.  A  model  might  retain  both  interpretations  until  one  or  the 
other  approaches  certainty.  A  model  can  help  analysts  determine  how  the  interpretation  fits  in 
with  everything  else  known  about  Ahmed.  For  example,  the  model  could  posit  that  Ahmed  was 
buying  bomb  materials,  but  through  a  motivation-based  simulation  might  determine  that  at  this 
point  in  time  the  insurgent  group  is  highly  unlikely  to  find  bombing  acts  to  their  advantage  (e.g.,  if 
the  French  populace  were  leaning  toward  pulling  out  of  Algeria,  a  bomb  attack  would  be 
interpreted  as  a  reason  to  stay  in  the  country).  The  MPC  might  then  accept  the  interpretation  that 
Ahmed  was  a  farmer,  and  refocus  its  data  collection  on  plans  for  strikes,  which  would  better 
achieve  the  goals  of  the  insurgency  at  the  present  stage  in  the  game. 

The  U.S.  Army  Intelligence  and  Information  Warfare  Directorate  (I2WD)  is  pursuing  an  MPC  project 
for  the  Intelligence  Community  that  implements  automated  feedback  loop  concepts.  Through  its 
Soft  Target  Exploitation  and  Fusion  (STEF)  program,  I2WD  has  previously  sponsored  projects  that 
can  serve  as  part  of  the  feedback  loop.  For  example,  Indra,  an  NLP  program,  can  enable  real-time, 
evolving  data  to  conform  to  a  model's  requirements.  Indra  generates  ontological  categories  of 
social  roles  and  role  relations  that  emerge  from  text  and  translates  them  into  the  particular 
ontology  of  a  computer  model  (Duong,  Stone,  Goertzel,  &  Venuto,  2010). 

One  aspect  often  neglected  in  sociocultural  modeling  is  the  capacity  of  the  model  to  absorb  data. 
Nexus  can  serve  in  another  part  of  a  feedback  loop  by  helping  a  model  conform  to  the  data.  Nexus 
uses  coevolutionary  pressure  to  explain  the  statistical  patterns  in  the  data  through  the  motivation 
of  the  agents,  so  that  the  simulation  can  recreate  the  same  vicious  and  virtuous  cycles  that  created 
the  data  in  the  real  world.  This  allows  interventions  to  be  tested  on  a  natural,  self-reinforcing 
system  (Duong,  Pearman,  &  Bladon,  2013). 

The  capacity  to  absorb  data  in  simulations  is  important  to  federations  of  models  because  it  enables 
participants  to  assume  new  consensus  states.  Data  processing  for  COA  analysis  should  account  for 
conflict  resolution  not  only  in  the  data  fed  to  simulations,  but  also  in  federated  models  that 
overlap.  Models  that  overlap  in  function  can  resolve  conflicting  results  using  voting  techniques  to 
achieve  consensus. 

Another  technique  for  arriving  at  consensus  related  to  competing  or  overlapping  functions  involves 
the  use  of  Nexus's  neural  network  for  constraint  satisfaction.  The  neural  network  can  place  positive 
links  between  event  relations  that  are  positively  correlated  in  the  real  world,  or  that  CREATE 
determines  to  be  consonant  with  theory.  Conversely,  the  neural  network  can  place  negative  links 
between  events  that  are  negatively  correlated  in  the  real  world,  or  that  CREATE  determines  to  be 
dissonant  with  theory.  Nexus  could  execute  the  network  to  determine  simulation  outputs  that  fit 
best  both  theory  and  the  real  world,  and  enter  them  back  into  all  federated  models  as  the 
consensus  state.  In  this  way,  the  computations  that  federated  models  make  in  an  MPC  system 
could  include  a  validation  step.  The  MPC  would  retain  the  most  valid  simulation  states  that  are 
both  consistent  with  each  other  and  consistent  with  theory. 
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9.  Science  and  Technology  Gaps  in  Data  Processing  for  COA  Analysis 

Industry,  government,  and  academia  have  all  recently  made  important  contributions  to  data 
processing  for  COA  analysis  in  the  areas  of  both  model  integration  and  formalization  of  COA 
concepts  in  data.  In  industry,  Impact  Computing's  XBM  integrates  models  to  represent  interactions 
among  strategizing  minds  in  general,  while  Sentia's  Senturion  integrates  a  specific  cognitive  model 
for  political  COAs.  In  government,  OSD  has  researched  coevolutionary  integration  of  learning 
mental  models  in  Nexus,  and  TRAC  has  preserved  uncertainty  in  data  and  translation  for  risk-based 
analysis  and  validation  using  SIMmiddleware.  Additionally,  ERDC  has  formalized  evidence  and 
sensemaking  data  in  CREATE,  and  I2WD  is  advancing  the  science  of  MPC  to  integrate  models  in  a 
feedback  loop.  The  Naval  Postgraduate  School's  Simulation,  Experiments  and  Efficient  Design 
(SEED)  Center  has  advanced  the  science  of  data  farming.  In  academia,  researchers  at  the  University 
of  Arizona,  Arizona  State  University,  and  Pennsylvania  State  University  have  improved  the 
representation  of  COA  extraction  from  text,  narrative  extraction  from  text,  and  a  coevolutionary 
framework  for  terrorist  learning,  respectively.  The  Department  of  Defense  (DoD)  has  supported 
many  of  these  advances.  However,  gaps  remain  in  the  underpinnings  of  how  to  use  all  of  these 
ideas  together  in  a  scientifically  valid  analysis. 

Although  the  tools  mentioned  in  this  chapter  offer  a  promising  start,  they  have  not  solved  the 
problem  of  valid  scientific  analysis  of  social  phenomena.  These  tools  begin  to  deal  with  the  most 
important  data  processing  gaps  for  COA  analysis— automation,  uncertainty,  and  risk-based 
analysis— but  do  not  yet  form  a  complete  program  that  has  the  blessing  of  the  scientific  community 
as  the  best  direction  to  take,  and  that  can  be  translated  into  formulas  for  use  for  operational 
personnel. 

In  the  past,  models  of  conventional  warfare  only  needed  a  limited  amount  of  pre-validated, 
physics-based  data.  In  contrast,  the  IW  models  of  today  demand  far  more  data  because  they  are 
based  on  the  scenario-specific  data  of  social  processes.  Little  scientific  research  has  focused  on 
automating  the  use  of  "big  data"  from  the  internet.  For  researchers  to  use  the  data  live  or  with  any 
scalability,  automated  tools  must  automatically  put  these  data  into  the  ontology  needed  by  the 
model.  Further,  models  should  be  modified  to  accept  data  that  they  do  not  anticipate.  Because 
social  science  is  scenario  specific,  the  social  model  should  not  predetermine  all  of  the  processes 
simulated;  instead,  the  data  should  determine  the  specific  processes  while  the  model  only  holds 
general  forms  of  social  processes  as  true.  The  capacity  to  automatically  absorb  data  for  models 
goes  hand-in-hand  with  the  capacity  to  automatically  represent  data  for  model  consumption.  Once 
tools  can  automatically  place  the  big  data  of  the  internet  in  the  correct  ontology  and  resolve 
conflicts  within  the  data,  the  data  base  can  become  more  reliable  than  the  individual  data  that 
comprise  it.  The  data  can  then  serve  as  both  a  training  set  for  model  creation  and  a  testing  set  for 
validation. 

Uncertainty  is  another  important  data  processing  gap  for  COA  analysis.  Non-validated  data  can 
potentially  conflict.  Closing  the  technological  gap  by  taking  advantage  of  the  amount  of  data  for  the 
resolution  of  conflict  in  the  data,  perhaps  in  a  type  of  automated  crowd  sourcing  technique  or 
other  voting  system,  will  permit  disparate,  conflicting  data  to  form  a  coherent  picture. 
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Conflict  occurs  not  only  in  data  that  go  into  models  but  also  in  data  produced  by  federated  models. 
Overlap  in  model  functionality  has  benefits  because  the  models  can  clarify  and  vote  on  conflicting 
areas.  Such  overlap  enables  users  to  test  CONOPS  for  robustness  against  every  plausible  model  of 
every  school  of  thought.  Data  processing  techniques  of  integration,  such  as  voting  systems, 
switching  in  and  out  in  proportion  to  pedigreed  credibility,  or  constraint  satisfaction  systems  can 
handle  functional  overlap  and  conflict  resolution  in  models. 

Furthermore,  risk-based  analyses  should  measure  uncertainty  and  conflict  in  the  data  coming  into 
the  models  and  going  out  of  the  models.  To  build  scientifically  valid  models  for  COA  analysis,  we 
must  advance  the  science  of  data  processing  to  make  data  available  for  validation.  We  also  need 
feedback  from  data  to  identify  the  most  promising  new  scientifically  based  model  technologies. 
The  confidence  that  analysts  have  in  the  data,  in  the  translation  of  the  data,  and  in  the  output  data 
should  be  kept  in  proportion  so  that  the  end  result  of  the  analysis  retains  a  probability.  The 
parameters  of  multiple  model  runs  should  be  chosen  according  to  their  likelihood  given  other 
parameters,  so  that  they  are  realistically  correlated.  The  DoD  as  a  whole  would  likely  have 
benefited  from  additional  investment  in  the  science  of  valid  social  analysis,  whether  focused  on 
models  or  on  data. 


10.  Technology  Transfer  of  Data  Processing  Techniques  for  COA  Analysis 

Today  knowledge  gaps  exist  regarding  processes  for  integrating  new  technologies  from 
government,  industry,  and  academia  into  operational  use.  This  gap  in  knowledge  of  the  science  of 
IW  analysis  hinders  operators  from  selecting  sociocultural  products  that  could  support  scientifically 
valid  analysis.  Operators  need  not  only  the  products,  but  also  guidelines  for  carrying  out  COA 
analysis  that  rest  on  both  scientific  theory  and  the  big  data  of  the  internet,  and  that  are 
comprehensible  to  non-scientists. 

Control  of  operational  analysis  that  uses  sociocultural  modeling  should  remain  in  the  hands  of 
scientists  until  the  scientific  community  comes  to  a  consensus  on  the  direction  that  scientific 
modeling  of  social  phenomena  should  take.  The  DoD  should  not  prematurely  choose  a  particular 
direction  or  constrain  the  science  through  standardized  procedures  of  analysis  before  the  natural 
process  of  scientific  argumentation,  experimentation,  and  validity  has  reached  a  consensus  from 
which  to  draw  a  procedure.  The  DoD  should  not  spend  6.4  (demonstation  and  validation)  funds 
until  6.1-funded  (basic  research)  products  are  mature,  no  matter  how  frustrating  the  wait. 
Scientific  competition  in  argumentation  and  experiment  should  ideally  enable  the  best  ideas  to 
bubble  to  the  top.  Short  of  that,  the  DoD  should  hire  scientists  to  spot  the  best  new  ideas,  rather 
than  having  operational  personnel  under  resource  constraints  control  social  model  analysis. 

Once  scientists  have  established  and  advocated  scientifically  sound  criteria  for  usage  of  individual 
methods,  and  schools  of  thought  have  explored  competing  paths  toward  scientific  analysis,  the 
scientists  can  lay  out  entire  maps  of  the  procedures  that  operations  personnel  should  perform  as 
part  of  scientifically  valid  analysis.  Without  careful  review  by  the  best  scientists  of  the  day, 
operational  analysis  of  social  phenomena  will  fail  to  meet  the  stringent  criteria  for  scientific 
validity. 
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1.  Introduction 

The  preceding  chapters  have  explored  the  complexities  of  understanding,  detecting,  and 
forecasting  sociocultural  phenomena  related  to  crucial  national  and  international  security  issues. 
They  showed  how  human  behavior— both  individual  and  societal— represents  a  confluence  of 
cultural,  political,  social,  economic,  and  historical  factors  interacting  in  a  dynamic  system.  This 
chapter  focuses  on  leveraging  that  information  to  develop  effective  mitigation  policies  that  can 
respond  to,  counteract,  or  utilize  these  sociocultural  behaviors. 

In  this  chapter,  we  describe  some  methods  and  tools  for  mitigation  analysis  through  computational 
modeling.  We  examine  several  computational  approaches  to  course  of  action  (COA)  development, 
analysis,  and  comparison,  noting  the  potential  for  positive  impact  on  the  military  decision-making 
process  (MDMP),  as  well  as  the  existing  technological  gaps  that  must  be  filled  to  achieve  effective 
sociocultural  mitigation.  This  chapter  expands  on  previous  discussions  of  mitigation  technologies 
presented  in  Kott  and  Citrenbaum  (2009),  which  focused  on  the  sociocultural  factors  and  the 
impact  on  military  operations.  The  particular  applications  presented  here  do  not  represent  an 
exhaustive  list  of  available  technology  solutions,  but  rather  a  sampling  of  computational 
approaches  that  may  facilitate  military  decision  making. 

The  data  themselves  are  central  to  the  problem  of  mitigation.  In  the  modern  information 
environment,  decision  makers  must  contend  with  an  overwhelming  quantity  of  data— ranging  from 
field  reports  and  sensor  readings  to  communication  signals  and  satellite  imagery— which,  as  earlier 
chapters  showed,  may  potentially  contain  indicators  of  extraordinarily  intricate  sociocultural 
phenomena.  Even  with  the  visualization,  data  processing,  and  modeling  techniques  already 
described,  the  amount  and  complexity  of  the  information  to  be  weighed  would  overwhelm  any 
human  analyst,  no  matter  how  talented  or  knowledgeable.  In  this  chapter,  we  discuss  how 
computational  modeling  can  relieve  some  of  this  burden,  allowing  decision  makers  to  synthesize 
and  integrate  the  models,  knowledge,  and  insights  from  the  understand,  detect,  and  forecast 
phases  of  analysis  to  provide  better  operational  awareness.  Using  many  methods  similar  to  the  big- 
data  analytics  employed  by  corporations  such  as  Amazon  or  Google,  analysts  can  exploit  this 
wealth  of  sociocultural  knowledge  to  support  decision  making  and  determine  the  most  robust  COA 
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in  many  critical  situations.  Of  course,  the  vast  domain  knowledge  and  experience  of  military  and 
civilian  experts  should  not  be  ignored;  the  computational  models  described  here  are  intended  to 
augment— not  automate— the  decision-making  process. 

In  Field  Manual  101-5  (U.S.  Army,  1997),  the  Department  of  the  Army  specifies  a  seven-step 
MDMP:  (1)  Receipt  of  mission,  (2)  Mission  analysis,  (3)  COA  development,  (4)  COA  analysis,  (5)  COA 
comparison,  (6)  COA  approval,  and  (7)  Orders  production.  Figure  1  outlines  these  steps,  illustrating 
both  the  commander's  and  the  staffs  roles  in  reaching  a  decision.  As  noted  in  Field  Manual  101-5, 
decision  making  requires  an  understanding  of  the  possible  consequences  of  a  COA  and  how  that 
COA  can  contribute  to  progress  from  the  current  situation  toward  the  stated  mission  goal.  The 
MDMP  is  designed  to  facilitate  comparison  of  multiple  COAs  and  analyze  them  in  light  of  possible 
enemy  COAs  or  decisions,  preventing  decision  makers  from  overlooking  possible  actions, 
outcomes,  or  critical  aspects  of  the  environment. 


NOTE  1 :  Commander  may 
conduct  phases  independently 
or  in  conjunction  with  staff. 


A 


NOTE  2:  Staff  coordination 


RECEIPT  OF  MISSION 

IS  CO 
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guidance 

WARNING  1 

ORDER 

MISSION  ANALYSIS 
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Estimate 
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♦  Commander’s 
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NOTE  3:  For  a  discussion 
of  rehearsals,  execution, 
and  assessment,  see 
Chapter  6  and  Appendix  G. 
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NOTE  4:  At  any  time  during 
execution  and  assessment, 
situation  may  require  the 
process  to  start  again. 


Figure  1.  Military  Decision-Making  Process  (MDMP).  Adapted  with  permission  from  U.S.  Army  Field 
Manual  10-5  (1997). 


Computational  models  can  expand  a  commander's  ability  to  examine  possible  options  and  account 
for  complex  behavioral  dynamics  when  considering  COAs  and  their  short-  and  long-range  impact. 
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Such  models  have  the  potential  to  augment  this  decision-making  and  mitigation  process  at  several 
steps,  outlined  in  red  in  Figure  1:  COA  development,  COA  analysis,  and  COA  comparison. 

COA  development  consists  of  producing  possible  COAs  for  further  analysis  and  comparison  (U.S. 
Army,  1997).  All  of  the  COAs  must  meet  some  minimal  standards  of  suitability  (i.e.,  adhere  to 
constraints  identified  by  the  commander),  feasibility,  acceptability  (in  terms  of  the  potential  cost- 
benefit  ratio),  distinguishability  (i.e.,  contains  characteristics  that  set  it  apart  from  other  COAs),  and 
completeness.  Exploratory  computational  modeling  utilizing  the  behavioral  models  and  forecasts 
previously  developed  for  situation  awareness  can  help  analysts  or  decision  makers  identify  COA 
variants  that  may  yield  better  outcomes  given  the  sociocultural  dynamics  of  the  current  context. 
Below  we  describe  several  computational  approaches  to  COA  generation,  such  as  abductive 
reasoning  and  artificial  intelligence  (Al)  planning. 

Typically,  COA  analysis  involves  wargaming  to  identify  the  strengths  and  weaknesses  of  a  COA  and 
major  decision  points  that  call  for  additional  detail  to  account  for  specific  situational  conditions 
(U.S.  Army,  1997).  COA  comparison  takes  this  process  a  step  further,  presenting  detailed  analysis  of 
which  COAs  are  effective  in  particular  contexts,  and  identifying  robust  options  that  have  a  high 
probability  of  success  in  light  of  the  opponent's  most  likely  (or  most  dangerous)  behaviors  (U.S. 
Army,  1997).  Computational  models  capable  of  utilizing  complex  knowledge  of  sociocultural 
phenomena  can  facilitate  both  of  these  aspects  of  the  MDMP.  Approaches  such  as  social  network 
analysis  and  tools  such  as  dynamic  systems  models  and  multi-agent  simulations  can  compare  the 
potential  outcomes  of  various  COAs  (both  human  and  model  generated)  under  a  wide  range  of 
plausible  conditions  to  support  the  MDMP. 


2.  Mitigating  Violent  Behavior  in  Pakistan 

Analysis  of  the  sociocultural  and  behavioral  landscape,  including  factors  such  as  potential  reactions 
of  adversaries  and  possible  roadblocks  resulting  from  civilian  cultural  perspectives,  is  essential  to 
the  success  of  the  MDMP.  Computational  models  can  support  the  MDMP  by  allowing  officers  and 
staff  to  exploit  large  quantities  of  sociocultural  data  and  expanding  their  capabilities  for  COA 
analysis  and  comparison. 

As  an  example,  we  present  a  complex  counterterrorism  scenario  where  a  decision-making  team 
must  develop  policies  to  mitigate  the  destabilizing  violence  perpetrated  by  the  terror  organization 
Lashkar-e-Taiba  (LeT)  in  Kashmir  and  India.  LeT  has  a  sophisticated  leadership  structure  and  an 
active  militant  wing  capable  of  executing  complex,  coordinated  attacks  throughout  the  Indian 
subcontinent  and  the  world.  While  the  organization  may  originally  have  sought  only  the  liberation 
of  Kashmir  and  establishment  of  an  Islamist  Pakistan,  since  the  attacks  on  September  11th,  2001, 
LeT  has  contributed  recruitment,  training,  personnel,  and  funding  to  the  global  jihadist  movement. 
The  group  plays  a  complicated  role  in  the  sociocultural  dynamics  in  Pakistan,  using  terrorist  tactics 
to  wage  jihad  in  India,  Afghanistan,  and  around  the  world,  while  being  deeply  integrated  into  the 
local  social  fabric  through  its  charitable  activities  and  provision  of  services  that  the  government 
cannot  maintain. 
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What  counterterrorism  policies  might  be  most  effective  in  discouraging  LeT's  use  of  violence?  How 
can  we  disrupt  or  counter  LeT  propaganda  conveyed  via  their  widespread  communications, 
education,  and  social  services  networks  without  threatening  the  security  of  the  Pakistani  citizens 
who  rely  on  these  services?  How  can  the  decision  makers  leverage  their  knowledge  about  the 
cultural  and  behavioral  dynamics  of  LeT  and  the  Pakistani  people  to  achieve  broader  goals  of 
security  and  stability  within  Pakistan  and  the  region?  Finding  answers  to  these  questions  will 
require  analysis  of  a  multifaceted  sociocultural  landscape,  augmenting  human  judgment  and 
expertise  with  large-scale  computational  modeling  and  simulation. 


3.  Goal-oriented  Automated  COA  Development 

Given  a  particular  mission  and  models  for  understanding,  detecting,  and  forecasting  sociocultural 
factors  within  the  operational  context,  the  first  phase  of  military  decision  making  involves 
developing  possible  COAs  that  could  feasibly  achieve  the  mission  goals.  Researchers  in  the  field  of 
Al  have  applied  two  main  approaches  to  developing  COAs  through  this  type  of  goal-oriented 
reasoning:  logical  abduction  and  Al  planning.  They  have  used  these  methods,  along  with  the 
predictive  models  discussed  in  Chapter  10,  to  construct  large-scale  decision  support  software 
systems  used  in  several  public  and  private  sector  decision-making  processes.  The  computational 
models  can  generate  possible  mitigation  policies,  assess  their  probability  of  effectiveness  and 
acceptability  relative  to  resource  and  feasibility  constraints,  and  elucidate  the  factors  that 
contribute  to  better  versus  worse  outcomes.  Below  we  briefly  describe  the  two  approaches, 
providing  some  examples  of  possible  applications  and  specific  tools  that  can  contribute  effectively 
to  mitigation. 

3.1.  Abductive  Reasoning 

In  many  everyday  situations,  people  must  interpret  a  set  of  observations,  using  their  past 
knowledge  of  how  the  world  works  to  explain  the  current  situation.  For  example,  a  homeowner 
wakes  up  in  the  morning  and  notices  that  her  lawn  is  wet.  From  prior  experience,  she  knows  that  if 
it  rains  during  the  night,  then  her  lawn  will  be  wet  in  the  morning.  Therefore,  she  can  conclude  that 
it  probably  rained  during  the  past  night,  and  she  may  decide  to  carry  an  umbrella  to  work  in  case 
the  bad  weather  continues.  That,  in  essence,  is  the  process  of  educated  guessing  that  all  people 
employ  countless  times  per  day:  given  some  behavioral  rule,  people  can  form  hypotheses  and  draw 
conclusions  about  entirely  new  situations.  This  type  of  reasoning  is  known  as  abduction  (not  to  be 
confused  with  kidnapping!)  Abductive  reasoning  centers  on  finding  the  most  logical  and  relevant 
explanation  for  a  set  of  facts.  This  mode  of  thinking  is  used  extensively  in  intelligence  analysis  and 
complex  decision  making. 

This  type  of  reasoning  does  not  limit  people  to  explaining  observed  facts;  they  can  use  the  same 
process  to  develop  possible  COAs  instead  of  hypotheses  (e.g.,  taking  an  umbrella  to  work). 
Basically,  using  abduction  for  COA  generation  involves  selecting  the  "optimal"  subset  of  actions 
that,  when  executed  in  the  current  situation,  will  achieve  the  stated  goal  with  the  highest 
probability,  taking  some  possible  constraints  into  account.  The  definition  of  optimality  depends  on 
many  factors,  such  as  the  current  operational  environment  or  limitations  on  time  or  resources. 
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Constraints  may  correspond  to  choosing  the  shortest  COA  (either  in  terms  of  the  number  of  distinct 
steps  or  the  total  time  to  execute  the  plan),  the  least  expensive  COA,  etc. 

The  Policy  Analytics  Generation  Engine  (PAGE)  develops  possible  COAs  using  both  exact  and 
approximate  abductive  computations,  while  the  APT  (Annotated  Probabilistic  Temporal)  Abduction 
System  (Simari  &  Subrahmanian,  2010;  Molinaro,  Sliva,  &  Subrahmanian,  2011)  centers  on  more 
complex  temporal  aspects  of  COA  generation.  The  Spatio-Cultural  Abductive  Reasoning  Engine 
(SCARE)  system  successfully  uses  abduction  to  identify  the  location  of  weapons  caches  and 
improvised  explosive  devices  (lEDs)  in  combat  zones,  allowing  for  development  of  specific 
mitigating  COAs  (Shakarian,  Subrahmanian,  &  Sapino,  2009;  Shakarian,  Subrahmanian,  &  Sapino, 
2011;  Shakarian,  Nagel,  Schuetzle,  &  Subrahmanian,  2011).  In  these  systems,  rather  than  start  from 
a  set  of  facts  or  observations,  users  set  a  goal— e.g.,  reduce  LeT  use  of  bombings  by  50%— and  find 
an  "explanation"  that  will  enable  progress  from  the  current  situation  toward  this  goal. 

If  the  strategic  goal  is  to  disrupt  LeT's  use  of  fedayeen  attacks  (suicide  attacks)  in  Kashmir,  the 
decision  maker's  objective  is  to  identify  ways  of  reducing  these  attacks,  rather  than  explain  how  or 
why  LeT  engages  in  such  attacks.  The  PAGE  system  utilizes  contextual  data  about  experience  with 
combat  in  the  region  and  sociocultural  data  regarding  LeT's  behavioral  dynamics  to  determine  the 
best  way  to  prevent  fedayeen  attacks.  The  software  outputs  a  set  of  actions  that,  when  executed, 
would  change  the  current  situation  to  one  in  which  fedayeen  attacks  have  been  reduced.  To  create 
deeper  insight  into  how  such  COAs  should  be  implemented,  the  APT  Abduction  System  adds 
temporal  analysis,  providing  the  optimal  time  frame  within  which  each  component  of  the  COA 
should  be  executed.  This  style  of  abductive  reasoning  assists  analysts  as  they  look  for  a  strategy 
that  can  induce  the  desired  effect,  rather  than  perform  data  analysis  to  describe  the  strategy  after 
the  fact. 

Abductive  reasoning  can  also  be  used  for  more  tactical-level  reasoning,  such  as  identifying  and 
destroying  enemy  weapons  caches  in  a  combat  zone.  Rather  than  explain  the  opponent's  attack 
strategy,  the  SCARE  system  utilizes  event  data  and  spatial  sociocultural  information  from  real 
combat  (for  example,  some  groups  tend  to  place  weapons  caches  in  close  proximity  to  IED  attack 
sites)  to  indicate  likely  locations  that  should  be  targeted  for  raids  or  further  investigation.  An 
extension  to  this  system  is  in  development  for  locating  other  types  of  high-value  targets  in  a 
counterinsurgency  environment,  particularly  Afghanistan. 

The  main  distinction  among  the  abductive  reasoning  frameworks  above  results  from  the  underlying 
data  needed  to  use  the  systems.  PAGE  requires  behavioral  data:  what  events  have  occurred,  how 
the  group  in  question  has  responded  to  past  strategies,  etc.  APT  Abduction  requires  time- 
annotated  behavioral  data,  covering  not  only  what  decisions  the  group  has  made,  but  also  the  time 
lags  between  changes  in  their  environment,  observed  behaviors,  and  possible  outcomes.  SCARE 
relies  on  spatial  sociocultural  information:  the  coordinates  of  past  attacks,  the  movement  patterns 
of  the  group,  etc.  The  choice  of  system  depends  on  the  available  data  and  the  problem  a 
commander  needs  to  solve.  For  example,  if  intelligence  indicates  that  LeT  will  attack  in  India  in  one 
week,  analysts  might  direct  APT  Abduction  to  generate  all  COAs  that  can  be  completed  in  under  a 
week. 
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Event  tracking,  mobile  phone  records,  satellite  imagery,  and  the  various  models  and  data 
processing  tools  described  in  Chapters  1,  5,  and  9  of  this  book  can  supply  much  of  the  necessary 
data.  However,  a  key  step  in  abductive  reasoning  and  COA  development  relies  on  appropriate 
processing  of  the  raw  data  to  make  them  usable  by  the  models  and  reasoning  systems.  PAGE,  APT 
Abduction,  and  SCARE  all  rely  on  a  data  representation  method  known  as  probabilistic  logic 
(Molinaro,  Sliva,  &  Subrahmanian,  2011;  Simari,  Dickerson,  Sliva,  &  Subrahmanian,  2012). 
Probabilistic  (and  temporal)  logic  has  been  extensively  studied  and  used  in  several  applications  as  a 
stochastic  representation  of  the  behavioral  dynamics  in  international  security  situations,  such  as 
terrorism,  drug  trafficking,  and  ethno-political  violence.  Fagin,  Halpern,  and  Megiddo  (1990), 
Hailperin  (1984),  Lukasiewicz  (1998),  Ng  and  Subrahmanian  (1992),  and  Nilsson  (1986)  give 
technical  details  of  probabilistic  logics;  Mannes,  Sliva,  Subrahmanian,  and  Wilkenfeld  (2008), 
Mannes,  Michael,  Pate,  Sliva,  Subrahmanian,  and  Wilkenfeld  (2008),  Sliva,  Martinez,  Simari,  and 
Subrahmanian  (2007),  and  Subrahmanian  et  al.  (2007)  describe  sociocultural  modeling  applications 
of  probabilistic  logics.  The  benefit  of  this  probabilistic  logic-based  approach  results  from  the 
balance  between  formal  reasoning— mathematical  models  that  computers  can  easily  manipulate— 
and  expressive  power  that  human  users  can  understand. 

Probabilistic  logic  explicitly  ties  behavioral  and  cultural  dynamics  to  facts  about  the  world 
environment,  relating  the  probability  of  actions  to  potential  contextual  factors,  such  as  the  political 
conditions  (e.g.,  existence  of  repressive  government  policies,  occurrence  of  elections,  corruption, 
etc.),  sociocultural  factors  (e.g.,  religious  beliefs,  ethnic  cleavages,  leadership  structures,  etc.),  or 
economic  conditions  (e.g.,  business  relationships,  economic  grievances,  macro-economic 
indicators,  etc.).  Such  rules  can  be  extracted  from  the  behavioral  or  spatial  data  previously 
mentioned  (Khuller,  Martinez,  Nau,  Simari,  Sliva,  &  Subrahmanian,  2007).  Figure  2  contains  a 
sample  of  a  probabilistic  logic  representation  of  some  behaviors  of  a  terror  organization  like  LeT, 
indicating  the  probability  of  violent  actions  as  a  result  of  situational  factors  in  the  environment 
(Subrahmanian,  Mannes,  Sliva,  Shakarian,  &  Dickerson,  2012).  The  first  line  says  that  if  the  group  is 
experiencing  conflict  with  another  rival  organization  (i.e.,  interorganizational  conflict),  then  the 
probability  that  it  will  use  kidnapping  as  a  strategy  lies  between  0.35  and  0.45. 


Probabilistic  Logic  Model  of  Terrorist  Behavior 


1.  kidnap :  [0.35, 0.45]  <—  interOrganizationConflicts. 

2.  kidnap :  [0.60, 0.68]  <—  unDemocratic  &  intemalConflicts. 


armedattacks :  [0.42,0.53]  <-  typeLeadership(strongSingle)  &  orgPopularity(moderate). 


Figure  2.  Example  of  probabilistic  logic  rules  describing  the  behavior  of  a  terror  organization. 
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Unfortunately,  abductive  COA  generation  can  be  computationally  very  intensive,  making  it  difficult 
to  incorporate  the  systems  mentioned  above  into  real-time  decision  making.  Researchers  have 
developed  a  more  scalable  distributed  framework  that  can  approximately  solve  for  the  top  k 
possible  COAs  (where  k  is  an  amount  determined  by  the  user)  (Simari,  Dickerson,  Sliva,  & 
Subrahmanian,  2012).  This  method  utilizes  parallel  computation  to  process  large  quantities  of 
sociocultural  information  simultaneously,  generating  several  different  COAs  with  different  starting 
assumptions.  For  example,  the  analyst  may  look  for  COAs  to  fit  a  hypothetical  situation  in  which 
Hafez  Saeed,  the  leader  of  LeT,  is  under  house  arrest,  or  COAs  that  would  prevent  him  from  finding 
robust  solutions  to  a  variety  of  contingencies.  Approximations  can  allow  decision  makers  to 
develop  many  alternative  mitigation  policies  in  a  more  tractable  timeframe. 

Another  flavor  of  abductive  reasoning  utilizes  Markov  logic  networks  (MLNs)  rather  than 
probabilistic  logic.  MLNs  resemble  probabilistic  logic  in  that  they  attempt  to  combine  the 
expressive  power  of  logic  (often  necessary  for  sociocultural  models)  with  the  uncertainty 
computations  of  probability  theory  (Kok  &  Domingos,  2009;  Richardson  &  Domingos,  2006). 
However,  MLNs  place  even  greater  emphasis  on  the  relational  aspects  of  the  environment, 
producing  a  network  structure  that  can  leverage  algorithms,  visualization  tools,  and  human 
expertise  developed  for  related  graphical  models  such  as  hidden  Markov  models,  dynamic  Bayesian 
networks,  or  conditional  random  fields.  MLNs  are  more  suitable  than  those  approaches  for 
sociocultural  domains  because  of  their  natural  representation  of  complex  behavioral  data,  such  as 
the  kind  developed  using  the  methods  in  Chapters  1,  5,  and  9. 

In  MLNs,  all  of  the  environmental  factors  (political  conditions,  sources  of  funding,  economic 
situations,  etc.)  and  behaviors  (kidnapping,  suicide  bombing,  etc.)  that  describe  a  group  are  nodes 
in  a  graph  and  are  connected  according  to  their  probabilistic  cause  and  effect  relationships  (Pearl, 
1988;  Richardson  &  Domingos,  2006).  Recall  the  (partial)  model  from  Figure  2  of  a  terrorist  group's 
violent  behavior  in  the  Middle  East.  Logic  rules  (1)  and  (2)  reveal  two  possible  relationships 
between  the  operational  context  and  the  occurrence  of  kidnapping.  In  the  first  case, 
interorganizational  conflict  is  a  potential  indicator  that  kidnapping  will  occur;  in  the  second  case  a 
combination  of  nondemocratic  group  leadership  and  internal  conflict  within  the  organization 
affects  the  likelihood  that  it  will  use  kidnapping  as  a  strategy.  An  MLN  can  represent  these 
possibilities  as  shown  in  Figure  3.  Because  interorganizational  conflict  is  related  to  kidnapping,  the 
analyst  draws  a  line  connecting  these  two  ideas.  Since  both  non-democratic  group  leadership  and 
internal  conflict  relate  to  kidnapping  as  well,  the  analyst  connects  these  with  the  action;  to  indicate 
that  these  two  factors  must  occur  at  the  same  time,  the  analyst  also  draws  an  edge  connecting 
them.  This  small  MLN  compactly  and  visually  captures  these  complex  relationships  between  the 
environment  and  observed  sociocultural  behaviors. 
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kidnap 


Figure  3.  Markov  logic  network  representing  factors  that  contribute  to  LeT's  decision  to  use 
kidnapping. 


Some  recently  developed  systems  can  apply  these  MLNs  to  probabilistic  abduction  (Kate  & 
Mooney,  2009;  Singla  &  Mooney,  2011).  These  processes  use  the  graphical  structure  to  facilitate 
efficient  analysis  and  identification  of  possible  policies.  For  example,  the  MLN  in  Figure  3  clearly 
contains  two  separate  sections  pertaining  to  kidnapping.  If  the  goal  is  to  reduce  kidnapping,  then 
an  analyst  can  quickly  identify  the  relationships  that  lead  to  the  lowest  likelihood  of  kidnapping.  In 
this  case,  rule  (1)  shows  that  fostering  intergroup  conflict  might  be  one  aspect  of  a  COA  to  counter 
the  organization's  militant  strategies.  Because  MLNs  are  easy  to  visualize,  decision  makers  in  the 
field  can  use  them  for  real-world  COA  generation.  Both  MLNs  and  probabilistic  logic  frameworks 
use  the  concept  of  abductive  reasoning  to  develop  possible  COAs  that  directly  utilize  the 
sociocultural  understanding  we  have  developed.  These  COAs  can  supplement  human-generated 
COAs  and  be  further  compared  and  analyzed  either  manually  or  with  additional  computational 
models  described  in  section  4  of  this  chapter. 

3.2.  Artificial  Intelligence  Planning 

Al  planning  also  explores  computational  models  for  automated  COA  discovery,  and  may  provide 
more  efficient  algorithms  or  intuitive  representations  in  some  tactical  domains.  Like  abduction,  Al 
planning  is  goal-based,  searching  for  possible  COAs  that  can  achieve  a  mission  given  the  current 
operational  environment  and  some  possible  constraints  on  time  or  resources. 

Particularly  relevant  to  the  complex,  dynamic  situations  confronting  military  decision  makers  is  the 
need  to  plan  under  uncertainty.  In  this  type  of  Al  planning,  models  generate  COAs  in  a  probabilistic 
domain  to  capture  uncertainty  about  either  the  outcomes  of  actions,  the  current  situation,  or  both. 
In  many  operational  environments  affected  by  sociocultural  dynamics  the  outcome  of  a  COA  (or 
part  of  a  COA)  may  be  uncertain— how  will  an  adversary  or  civilian  population  react  to  a  particular 
action?— but,  more  important,  analysts  may  be  uncertain  about  the  exact  nature  of  the  human 
terrain.  Their  sociocultural  understanding  will  provide  information  about  the  possible  behaviors 
that  may  have  an  impact  on  a  COA,  but  the  complexity  of  social  processes  means  that  the  resulting 
models  may  be  incomplete,  noisy,  and  uncertain.  There  may  simply  be  some  parts  of  the 
operational  situation  that  military  units  cannot  (or  do  not)  monitor.  To  account  for  these  gaps  in 
knowledge,  analysts  can  use  a  computational  COA  discovery  model  based  on  partially  observable 
Markov  decision  processes  (POMDPs)  (Kaelbling,  Littman,  &  Cassandra,  1998;  Kobolov  &  Kobolov, 
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2012).  POMDPs  have  been  utilized  in  many  real-world  applications:  Yost  and  Washburn  (2000) 
developed  methods  for  solving  large-scale  military  allocation  problems  with  POMDP  algorithms, 
and  MIT  Lincoln  Laboratories  and  the  National  University  of  Singapore  are  exploring  a  variant  of 
POMDPs  as  a  method  for  automated  collision  avoidance  in  unmanned  aerial  vehicles  (Bai,  Hsu, 
Kochenderfer,  &  Lee,  2012). 

The  immense  complexity  of  the  problem  can  make  exact  algorithms  for  solving  POMDPs  and 
generating  possible  COAs  intractable  in  real-world  cases.  Current  research  in  Al  planning  has 
developed  several  heuristic  approaches  that  may  make  POMDP  models  more  accessible  to  users 
who  need  to  incorporate  sociocultural  knowledge,  allowing  them  to  access  these  decision-making 
aids  in  near-real  time  using  the  limited  computing  resources  available  in  the  field  (Ross,  Pineau, 
Paquet,  &  Chaib-draa,  2008).  Heuristic  approaches  make  trade-offs  between  accuracy  and 
efficiency,  meaning  that  developers  must  give  careful  consideration  to  the  role  of  human  decision 
makers  in  ensuring  accuracy  of  these  systems. 

During  COA  generation  it  is  often  crucial  to  identify  the  low-level  tasks  that  must  be  accomplished 
to  achieve  a  broader  mission  goal.  This  type  of  planning  can  utilize  an  automated  planning 
approach  known  as  hierarchical  task  networks  (HTNs),  which  explicitly  decomposes  the  COA  into 
smaller  subtasks.  For  example,  a  decision  maker  may  want  to  formulate  a  high-level  plan  to  reduce 
the  number  of  fedayeen  attacks  in  Kashmir  by  LeT  militants.  Using  an  HTN  system,  the  decision 
maker  might  decompose  this  problem  according  to  the  hierarchy  in  Figure  4.  Reducing  fedayeen 
attacks  has  two  components:  ending  Pakistani  military  support  of  the  LeT  militants  and  disrupting 
the  internal  cohesion  of  the  organization.  The  second  component  consists  of  two  pieces: 
encouraging  the  LeT  leadership  to  resign  and  causing  the  group  to  splinter.  Of  course,  both  of  these 
subtasks  are  very  broad  in  and  of  themselves,  and  in  a  real-world  HTN  planner  they  would  be 
further  divided  into  lower  level  operations  to  accomplish  each  of  these  goals. 


Figure  4.  HTN  for  reducing  LeT  fedayeen  attacks.  The  network  decomposes  the  main  goal  into 
component  parts,  each  of  which  might  be  further  subdivided  into  several  possible  COAs. 
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HTNs  can  automatically  explore  different  possibilities  or  combinations  of  actions  to  construct 
possible  COA  options  that  address  the  subtasks  in  the  network.  Because  they  portray  the  structural 
relationships  among  different  aspects  of  a  COA,  planning  focuses  only  on  COAs  that  consist  of 
actions  that  help  to  achieve  a  subtask.  This  directed  approach  leads  to  computation  that  can  be 
more  efficient  for  COA  generation  (Nau,  llghami,  Kuter,  Murdock,  Wu,  &  Yaman,  2003). 

The  SOCAP  (System  for  Operations  Crisis  Action  Planning)  HTN  framework,  created  by  SRI 
International  and  tested  in  the  field,  uses  a  variety  of  data,  such  as  threat  assessments,  terrain 
analysis,  force  deployment,  transport  capabilities,  enemy  behavioral  factors,  and  operational 
constraints,  to  find  possible  plans.  SOCAP  decomposes  a  library  of  military  actions  into  subgoals, 
allowing  the  user  to  select  a  particular  objective  (e.g.,  disrupt  LeT  communications),  define  the 
constraints  on  the  environment  (e.g.,  input  the  necessary  sociocultural,  political,  geographical,  etc., 
data),  and  compute  possible  COAs  for  tactical  situations,  such  as  determining  equipment 
configurations  or  emergency  response.  The  Army  has  also  investigated  HTN  planning  for  use  in 
small  unit  operations  (Bienkowski,  des  Jardins,  &  Desimone,  1994;  Tate,  Levine,  Jarvis,  &  Dalton, 
2000).  In  this  application  SOCAP  automatically  translates  the  OPORDs  (Operations  Orders) 
produced  by  high  levels  of  the  military  into  an  HTN.  Users  can  then  choose  from  a  menu  of  possible 
objectives  and  use  the  system  to  generate  plans  for  various  tactical  environments,  such  as  a 
counterinsurgency  scenario.  However,  few  of  the  most  recent  advances  in  Al  planning  have  so  far 
been  incorporated  into  deployed  systems. 

Another  subset  of  Al  planning  particularly  relevant  to  COA  discovery  in  a  complex  sociocultural 
domain  is  the  field  of  mixed-initiative  planning  and  scheduling  (MIPAS)  (Bernstein,  Beranek,  Inc,  & 
Mcdermott,  1996).  MIPAS  combines  the  efficiency  and  data  processing  capabilities  of  automated 
planners  with  the  specialized  human-in-the-loop  knowledge  of  expert  users.  Where  POMDP  or  HTN 
planners  depend  on  complicated  probabilistic  algorithms  that  can  be  incomprehensible  to  actual 
users,  decision  makers  may  find  MIPAS  approaches  more  intuitive;  rather  than  wait  for  the  result 
of  a  systematic  search,  the  user  directs  COA  generation  by  dynamically  indicating  to  the  system 
those  factors  that  are  most  relevant  and  must  be  explored  or  by  choosing  partial  COAs  that  are 
most  useful.  As  the  situation  on  the  ground  changes,  the  user  can  simply  click  the  appropriate 
options  on  the  system  to  indicate  a  change  in  preferences.  Several  MIPAS  planners  have  been 
developed  for  specific  applications,  such  as  activity  plans  for  the  Mars  Rover  (Bresina,  Jonsson, 
Morris,  &  Rajan,  2005)  and  an  interactive  COA  planning  system  based  on  the  Air  Force  Research 
Laboratory's  Causal  Analysis  Tool  (CAT)  (Kutur,  Nau,  Gossink,  &  Lemmur,  2004).  This  system  allows 
users  to  iteratively  create  plans  using  CAT's  causal  networks  for  analysis  and  comparison.  In 
addition,  research  into  general  MIPAS  frameworks,  such  as  the  Goal  Transformations  (GTrans) 
Planner,  can  potentially  be  adapted  to  COA  generation  by  using  available  sociocultural  data  as 
input  and  domain  experts  as  the  interactive  users  (Cox,  2003;  Zhang,  Cox,  &  Immaneni,  2002). 
GTrans  has  already  been  applied  to  emergency  response  situations  (Immaneni  &  Cox,  2004). 

All  of  the  above  methods  can  aid  decision  makers  in  identifying  possible  COAs  in  a  particular 
operational  environment.  Computational  decision  support  systems  (DSSs)  can  enable  decision 
makers  to  tap  the  vast  sociocultural  information  from  the  understanding,  detection,  and 
forecasting  phases  that  may  be  too  complex  for  experts  to  analyze  manually.  In  addition,  because 
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such  computational  methods  are  not  subject  to  the  same  cognitive  biases  or  stressful  situations  as 
human  experts,  these  systems  make  it  possible  to  automatically  generate  COAs  that  might 
otherwise  have  been  overlooked. 


4.  Dynamic  Simulations  for  COA  Comparison  and  Analysis 

Once  analysts  have  developed  several  candidate  COAs,  either  human  or  automatically  generated, 
the  next  phases  of  the  MDMP  involve  comparing  the  possible  COAs  and  analyzing  their  relative 
strengths,  weaknesses,  and  likelihoods  of  success.  Computational  models  can  facilitate  comparing 
and  analyzing  COAs,  allowing  decision  makers  to  consider  a  much  broader  range  of  possible  social 
dynamics  that  could  influence  COA  effectiveness.  Computational  modeling,  such  as  dynamic 
simulation  systems,  can  help  identify  the  most  robust  COA,  the  one  that  will  best  balance  resources 
and  feasible  success  in  a  complicated  sociocultural  context  (Bankes,  2002;  2010).  The  following 
subsections  detail  several  of  these  methods,  focusing  on  the  ways  that  models  developed  for 
sociocultural  understanding,  detecting,  or  forecasting  can  be  repurposed  and  combined  for 
comprehensive  COA  analysis  and  comparison. 

Dynamic  simulation  systems  harness  computational  power  to  determine  the  most  probable 
outcome  of  a  COA,  using  the  behavior  of  the  model  to  analyze  complex  sociocultural  factors  and 
interactions  that  may  have  an  impact  on  COA  success.  These  simulations  effectively  comprise  a 
virtual  laboratory  where  analysts  can  capture  the  full  decision-making  processes  of  adversaries  or 
societies  of  interest  and  evaluate  how  effectively  a  COA  can  mitigate  such  behaviors.  By  examining 
how  behaviors  evolve  over  time,  they  can  capture  emergent  or  adaptive  behavior  that  may 
influence  seemingly  robust  COAs.  For  example,  a  COA  intended  to  induce  internal  conflict  within 
LeT  may  achieve  this  ultimate  outcome,  but  during  the  course  of  its  application  lead  to  increased 
instability  in  northern  Pakistan.  Thus,  choosing  the  most  robust  COA  does  not  depend  only  on  the 
end  result,  but  also  on  the  intermediate  effects. 


4.1.  Game-theoretic  and  Cognitive  Simulation  Models 

Chapter  2  in  this  book  presented  several  variations  on  rational  choice  and  game-theoretic  models. 
Such  models  can  also  be  used  to  analyze  possible  mitigation  options,  allowing  decision  makers  to 
explore  the  effects  of  particular  actions  on  the  decisions  and  behaviors  of  adversaries  or  societies 
(Myerson,  1991;  Raiffa,  2002;  Sandler  &  Arce,  2003).  The  main  organizing  principle  for  these 
models  is  rational  choice  theory,  whereby  actors  make  decisions  to  maximize  potential  benefits 
and  minimize  costs  based  on  their  preferences  and  the  expected  actions  of  other  actors. 

Recent  work  on  dynamic  game-theoretic  models  incorporates  bounded  rationality,  relaxing  the 
concept  of  rational  behavior  and  including  elements  from  cognitive  or  behavioral  psychology 
(Cramerer,  2003;  Gigerenzer  &  Selton,  2002).  Simulation  analyses  can  use  such  game-theoretic 
models  in  comparing  the  adversary  behavior  that  will  occur  as  a  result  of  various  potential  COAs. 
For  example,  a  system  developed  by  scientists  at  the  Duke  University  Institute  for  Homeland 
Security  Solutions  combines  the  adversarial  aspects  of  game  theory  with  the  uncertainty  and 
psychology  of  risk  analysis,  recognizing  that  behavior  may  change  in  probabilistic  ways  depending 
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on  conditions  of  risk  (Banks,  2009).  Using  Bayesian  statistics,  the  system  induces  a  probability 
distribution  over  a  possible  set  of  outcomes  at  each  time  in  the  simulation,  describing  the  possible 
costs/benefits  of  a  given  COA  relative  to  a  particular  enemy  strategy. 

In  a  dynamic  simulation,  users  can  initialize  the  model  with  data  from  the  current  operational 
environment  and  use  the  probability  distribution  to  examine  all  possible  combinations  of  friendly 
and  opponent  COAs  at  each  time  point.  This  gives  the  system  enough  information  to  compute  a 
cost-benefit  analysis  for  each  possible  COA  under  consideration,  comparing  the  relative  probability 
of  success  in  light  of  the  most  likely  adversary  behavior  (Banks,  2009).  Repeating  this  simulation 
multiple  times  can  generate  further  support  for  a  decision,  enabling  comparison  of  COAs  based  on 
the  number  of  times  the  system  chose  each  one  as  the  optimal  solution.  The  more  frequently  a 
COA  is  chosen,  the  more  robust  it  may  be  as  a  mitigating  policy.  This  risk-based  framework  has 
been  used  in  real-world  bioterrorism  scenarios  to  analyze  possible  defensive  COAs  (Banks  & 
Anderson,  2006). 

A  similar  simulation  approach  explicitly  incorporates  the  psychological  dynamics  of  threat 
perception  into  game-theoretic  simulations.  Behavioral  psychology  and  political  science  have  noted 
the  impact  of  threat  on  decision  making  (Huddy,  Feldman,  Taber,  &  Lahav,  2005;  Pyszczynski, 
Solomon,  &  Greenberg,  2003;  Schelling,  1960),  where  actors  suddenly  and  drastically  change  their 
preferences  under  fear  or  stress  (Kahneman  &  Renshon,  2006;  Kahneman,  Slovic,  &  Tversky,  1982). 
Researchers  at  Northeastern  University  have  developed  a  game-theoretic  simulation  model  based 
on  decision  making  under  threat  (Kohentab,  Pierce,  &  Ben-Perat,  2012;  Pierce,  Boulay,  & 
Malioutov,  2011;  Pierce,  Boulay,  &  Malioutov,  2012).  At  every  time  point  in  the  simulation,  the 
model  computes  the  threat  perception  for  each  friendly  and  enemy  group  based  on  events  in  the 
previous  time;  for  example,  if  an  attack  occurred  in  that  period,  threat  perception  may  have 
increased.  The  model  incorporates  these  psychological  responses  to  conflict  events  into  a  game- 
theoretic  framework  by  determining  each  actor's  preferences  at  each  time  point.  This  framework 
enables  analysis  of  the  efficacy  of  various  COAs  by  noting  the  level  of  threat  they  induce 
throughout  the  simulation  and  the  persistence  or  desistance  of  violent  strategies  over  time. 

While  these  approaches  can  enable  analysis  of  how  each  possible  COA  will  interact  with  an 
adversary's  cultural  and  behavioral  processes,  it  is  not  always  possible  to  model  truly  emergent  or 
adaptive  social  behavior  with  game-theoretic  approaches  because  of  their  strict  mathematical 
representation.  The  following  sections  describe  multi-agent  and  complex  network  simulation 
models  that  can  analyze  COAs  in  light  of  dynamic  and  nonlinear  behavioral  models. 

4.2.  Complex  Systems  and  Network  Simulation  Models 

Complex  systems  science  is  the  study  of  emergent  behaviors  within  a  large,  interconnected  system. 
Dynamic  network  simulation  models  offer  one  representation  for  complex  systems  approaches. 
Chapter  2  presented  a  detailed  discussion  of  social  network  models  and  social  influence  network 
theory.  Decision  makers  can  use  many  of  these  same  models  for  COA  analysis  to  identify  how 
reactions,  perceptions,  or  ideas  may  spread  through  a  population  during  implementation  of  a  COA. 
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In  addition,  several  network  simulations  have  been  developed  specifically  for  COA  analysis,  utilizing 
structural  information  about  sociocultural  factors  to  map  the  influence  of  a  COA.  Pythia,  a 
computational  modeling  tool  developed  by  researchers  at  George  Mason  University,  uses  a  timed 
influence  network  model  to  graphically  represent  causal  relationships  in  a  social  process 
(Wagenhals  &  Levis,  2007).  In  this  network,  root  nodes  (nodes  with  no  incoming  edges)  indicate  a 
particular  COA,  including  the  time  sequence  of  its  component  actions,  while  leaf  nodes  (nodes  with 
no  outgoing  edges)  represent  possible  final  outcomes.  The  interior  part  of  the  network  models  the 
flow  of  influence  and  events,  where  a  direct  connection  indicates  a  causal  link,  each  with  an 
associated  conditional  probability  distribution.  The  system  allows  users  to  incorporate  time  lags 
into  the  COA  events  to  see  how  the  network  will  evolve  over  time. 

Figure  5  traces  the  effects  of  a  COA  where  (a)  LeT's  communications  are  interrupted,  (b)  LeT  camps 
are  disrupted,  and  (c)  the  Pakistani  military  ceases  to  support  the  group.  This  analysis  examines  the 
impact  of  the  COA  on  the  internal  cohesion  of  LeT,  which  in  turn  has  a  causal  impact  on  LeT's 
executing  an  armed  attack.  (Of  course,  in  a  real  model  the  single  node  representing  LeT  internal 
conflicts  could  be  a  large  social  network  representing  the  social  dynamics  of  the  group  and  its 
constituents.)  As  each  element  of  the  COA  goes  into  effect  according  to  the  time  delay,  the 
probabilities  of  various  aspects  of  LeT's  internal  conflicts  change;  this  effect  propagates  to  the 
likelihood  of  LeT's  carrying  out  an  armed  attack.  The  Pythia  simulation  system  makes  it  possible  to 
map  out  the  likelihood  of  specific  outcomes— such  as  LeT's  use  of  violence— as  a  result  of  social 
processes  that  evolve  as  a  COA  progresses.  This  facilitates  comparison  between  various  COAs  or 
different  temporal  variations  of  the  same  COA  (Pachowicz,  Wagenhals,  Pham,  &  Levis,  2007; 
Pythia:  Timed  Influence  Net  Modeler;  Wagenhals  &  Levis,  2007). 


Figure  5.  Influence  network  indicating  the  impact  of  a  possible  COA  on  the  likelihood  of  LeT  internal 
conflicts  and  behavior. 
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The  Organization  Risk  Analyzer  (ORA),  developed  by  the  Computational  Analysis  of  Social  and 
Organizational  Systems  lab  at  Carnegie  Mellon  University,  is  another  dynamic  network  analysis  and 
simulation  tool  that  can  facilitate  COA  analysis  (Carley  &  Pfeffer,  2012;  Moon,  Carley,  &  Levis, 
2008).  ORA  models  the  internal  social  and  structural  dynamics  within  an  organization  or 
sociocultural  group.  To  use  this  tool  for  COA  analysis,  analysts  must  link  the  elements  of  the  COA  to 
relevant  parts  of  a  dynamic  sociocultural  network  created  for  either  understanding  or  forecasting 
in  this  domain  (see  Chapters  2  and  10),  similar  to  the  Pythia  example  in  Figure  5.  ORA  can  then 
analyze  the  impact  of  the  COA  in  terms  of  the  vulnerabilities  or  risks  introduced  into  the  network, 
how  the  sociocultural  system  changes  over  time,  and  the  interaction  among  various  different 
networks  of  interest  (Carley  &  Pfeffer,  2012;  Carley,  Pfeffer,  Reminga,  Storrick,  &  Columbus,  2012; 
Moon,  Carley,  &  Levis,  2008).  Depending  on  the  goals  of  the  mission,  decision  makers  can  compare 
these  COAs  to  determine  the  most  robust  option. 

Basic  research  advances  in  complex  science  make  COA  analysis  for  mitigation  a  promising  field  for 
modeling  (Barabasi,  2002;  Bar-Yam,  2003).  For  example,  several  other  models  have  been  applied  to 
COA  analysis  and  comparison,  such  as  the  New  England  Complex  Systems  Institute  model  for 
comparing  possible  intervention  policies  to  stem  the  production  of  poppy  in  Afghanistan,  or 
support  counterterrorism  efforts  and  explore  how  policies  with  a  negative  perception  may  increase 
instability  or  volatility  (Fostel  &  Geanakoplos,  2012;  Lewling  &  Sieber,  2007;  Widener,  Bar-Yam, 
Gros,  Metcalf,  &  Bar-Yam,  2012). 


4.3.  Multi-agent  Simulations 

Chapters  2  and  10  discussed  the  application  of  agent-based  models  to  understanding  and 
forecasting  behavior  in  sociocultural  systems.  These  models  represent  a  complex  system  as  a 
collection  of  autonomous  decision-making  agents  that  can  execute  behaviors  according  to  specific 
sociocultural  interactions  (Bonabeau,  2002).  The  same  modeling  and  simulation  frameworks  that 
facilitate  sociocultural  understanding  or  forecasting  can  also  be  used  for  COA  analysis,  testing  the 
impact  of  potential  COAs  in  an  interactive  and  interconnected  social  environment. 

One  simulation  tool  under  development  for  agent-based  COA  analysis  is  the  Course  of  Action 
Analysis  with  Radio  Effects  Toolbox  (CARET),  created  by  RAM  Laboratories  (McGraw,  Shao, 
Mumme,  &  Macdonald,  2009).  While  still  in  the  prototype  phase,  CARET  has  provided  an  agent- 
based  analysis  of  complex  behavior  in  a  military  mission  context  through  dynamic  COA  simulations 
that  facilitate  real-time  decision  making.  CARET  enables  users  not  only  to  evaluate  the  current  COA 
in  real  time,  but  also  to  simulate  and  compare  new  possible  COAs  to  determine  if  a  change  of 
course  might  be  more  effective.  This  system  is  particularly  designed  to  aid  in  replanning  efforts: 
that  is,  updating  a  COA  in  light  of  new  information  or  events  in  the  operational  environment.  For 
example,  during  the  execution  of  a  policy  to  disrupt  operations  at  LeT's  main  training  compound 
near  Muridke,  Pakistan,  the  social  situation  may  become  unstable  due  to  arrests  of  LeT  members 
elsewhere  in  the  country.  Modelers  can  collect  the  new  behavioral  dynamics  of  this  situation  and 
incorporate  them  into  CARET  to  further  analyze  COA  viability. 
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Research  by  Laval  University  in  Canada  has  also  explored  explicit  use  of  Multi-Agent  Geosimulation 
(MAGS)  to  support  qualitative  analysis  of  different  COAs  (Haddad  &  Moulin,  2008).  This  system  is 
designed  specifically  to  facilitate  the  type  of  "what  if"  analysis  necessary  for  comparing  the 
outcomes  and  side  effects  of  various  COAs  in  a  complex  sociocultural  environment.  Analysts  can 
use  the  implanted  MAGS-COA  to  simulate  the  execution  of  a  COA  in  a  particular  virtual  geographic 
environment  that  describes  the  geographic  features  of  the  operational  theater  and  can  change 
dynamically  along  with  various  sociocultural  processes.  Users  can  explore  various  COAs  and 
assumptions  through  the  agent-based  simulations,  modeling  how  the  environment  evolves  over 
time  as  the  COA  is  implemented.  The  MAGS-COA  system  also  directly  facilitates  COA  comparison, 
producing  a  qualitative  report  at  the  end  of  the  simulation  runs  that  documents  the  impacts  of 
various  potential  COAs  and  their  relative  success  in  achieving  particular  mission  goals  (Haddad  & 
Moulin,  2008).  Similar  analyses  can  also  be  run  using  the  multi-agent  understanding  and 
forecasting  simulations  described  in  Chapters  2  and  10. 


5.  Decision  Support  Systems 

The  previous  sections  have  briefly  introduced  some  computational  modeling  methods  with  the 
potential  to  augment  the  MDMP,  allowing  users  to  leverage  large  quantities  of  sociocultural  data, 
models,  and  forecasts  in  conjunction  with  their  own  specialized  knowledge.  However,  many  of  the 
techniques  described  above  are  still  being  explored  at  the  basic  research  or  prototype  level.  To 
enter  operational  use  for  real-world  COA  discovery,  analysis,  and  comparison,  these  computational 
models  must  be  integrated  into  comprehensive  applications  known  as  DSSs.  A  DSS  is  typically 
defined  as  an  interactive  computer-based  system  intended  to  help  decision  makers  use 
communications  technologies,  data,  documents,  knowledge  and/or  models  to  identify  and  solve 
problems,  complete  decision  process  tasks,  and  make  decisions  (Druzdzel  &  Flynn,  2002;  Shim  et 
al.,  2002).  Developers  have  created  DSSs  since  the  1970s  for  a  wide  range  of  domains— medicine, 
business,  air  traffic  control,  military,  etc.— that  require  complex  decision  making  or  logistic 
planning.  Recent  technological  advances  have  led  to  a  resurgence  in  DSS  research  and 
development,  with  many  promising  applications.  The  military  has  long  utilized  DSSs  for  non¬ 
battlefield  decisions,  such  as  personnel  housing  plans  in  the  Army  and  the  U.S.  Transportation 
Command's  Regulating  and  Command  &  Control  Evacuation  System  (TRAC2ES),  developed  by  Booz 
Allen  Hamilton,  for  medical  evacuation  logistics  (Booz  Allen  Hamilton,  2003).  Several  other  DSS 
frameworks  have  been  developed  for  military  decision  making,  but  have  not  been  widely  deployed. 

Researchers  from  the  University  of  Minnesota,  the  U.S.  Air  Force  Academy,  the  National 
Aeronautics  and  Space  Administration,  the  Department  of  the  Army,  and  the  University  of  Illinois 
have  developed  a  suite  of  decision  support  tools  for  various  phases  of  the  MDMP,  including 
generation  and  analysis  of  COAs  and  enemy  COAs  (ECOAs).  FOX-GA,  an  Al  planning-based  system, 
uses  genetic  algorithms  to  discover  COAs  for  military  maneuvering,  providing  users  with  a  range  of 
possible  options  in  a  time-constrained  environment  (Schlabach,  Hayes,  &  Goldberg,  1998).  To 
complement  these  COAs,  the  Weasel  system  uses  a  behavioral  model  of  an  enemy  or  other  third 
party  to  identify  possible  ECOAs  that  could  result  from  each  possible  COA  (Larson  &  Hayes,  2005; 
Hayes  &  Ravinder,  2003;  Ravinder  &  Hayes,  2003).  Weasel  implements  a  mixed-initiative  planning 
interface  and  can  facilitate  a  "what  if?"  style  of  COA  analysis.  These  two  systems  are  designed  to  be 
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used  in  conjunction  with  the  CoRaven  (or  perhaps  other  modeling  or  forecasting  tools)  system  for 
intelligence  analysis  (Jones,  et  al.,  1998).  Together,  these  individual  DSSs  can  constitute  a  more 
complete  decision  support  framework  for  mitigation. 

The  Massachusetts  Institute  of  Technology  (MIT)  Lincoln  Laboratory  is  also  working  on  a  system  for 
integrated  sensing  and  decision  support,  with  specific  research  programs  in  a  variety  of  military 
applications  ranging  from  special  operations  to  missile  defense  to  air  traffic  control  (Senne  & 
Condon,  2007).  This  framework  integrates  remote  sensing  data,  which  may  include  sociocultural 
sensors  such  as  social  media  or  the  outputs  of  various  forecasting  or  detection  models,  into  a 
system  for  decision  support,  simplifying  the  decision-making  process  and  incorporating  otherwise 
unavailable  information. 

Typically,  DSSs  perform  most  effectively  in  augmented  decision  making,  facilitating  human 
decisions  rather  than  fully  automating  the  analysis  and  final  choice  of  COAs.  These  tools  can  help 
speed  up  the  MDMP  at  crucial  points  or  permit  a  deeper  understanding  of  the  operational 
environment,  particularly  in  cases  where  complex  sociocultural  factors  must  be  taken  into 
consideration.  This  is  especially  true  in  the  sociocultural  domain,  where  even  large-scale  models 
still  contain  high  levels  of  uncertainty  (Bankes,  Popper,  &  Lempert,  2006).  By  providing  for  a  more 
efficient  and  complete  MDMP,  a  DSS  can  help  reduce  the  burden  of  complex  decision  making  in 
very  high-stakes  situations.  Of  course,  decision  makers  may  be  tempted  to  over-rely  on  a  DSS; 
adequate  training  and  continual  assessment  are  necessary  to  ensure  correct  utilization  of  a 
deployed  DSS. 

One  of  the  other  potential  benefits  of  using  a  DSS  for  mitigation  stems  from  the  ability  of  such  tools 
to  "de-bias"  COA  generation  or  analysis  (Davis,  Kulick,  &  Egner,  2005).  Unaffected  by  the 
preconceptions  inherent  in  human  decision  making,  DSSs  can  promote  objective  decision  making 
by  generating  COAs  that  a  particular  decision  maker  would  otherwise  not  have  considered  or  by 
analyzing  possible  COAs  from  an  impartial  perspective.  However,  some  studies  have  shown  that 
use  of  computational  decision  aids  can  actually  worsen,  or  at  the  very  least  fail  to  combat,  existing 
cognitive  biases  (Davis,  Kulick,  &  Egner,  2005).  In  spite  of  the  potential  drawbacks  of  DSSs,  these 
computational  approaches  to  decision  making  can  provide  an  excellent  resource  for  sociocultural 
mitigation,  making  the  MDMP  more  efficient  and  allowing  a  richer  analysis  of  the  operational 
environment. 


6.  Operational  Challenges  of  Computational  Mitigation  Models 

As  with  all  new  technologies,  the  devil  is  always  in  the  details.  Implementing  effective  DSSs  or 
other  computational  models  for  mitigation  in  sociocultural  environments  remains  a  complicated 
technological,  engineering,  and  training  challenge. 

From  a  research  and  development  perspective,  research  must  still  fill  many  gaps  before  any  of  the 
above  mitigation  models  can  be  deployed  for  operational  use.  Currently  the  MDMP  can  be  very 
slow,  but  commanders  are  under  extreme  pressure  to  quickly  find  robust  and  effective  COAs  in 
severely  time-constrained  environments.  While  the  computational  models  described  above  allow 
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users  to  consider  far  more  data  regarding  sociocultural  dynamics  than  a  human  alone  could 
analyze,  the  high  complexity  of  the  data  means  that  these  models  may  not  necessarily  arrive  at  a 
solution  faster. 

In  many  cases,  these  computational  models  and  simulations  not  only  require  excessive  running 
time,  but  also  consume  massive  amounts  of  computing  resources  to  process  the  large  quantity  of 
data  and  complexity  of  the  sociocultural  dynamics.  While  high-performance  and  distributed 
computing  platforms,  such  as  Apache  Hadoop  (Apache,  2012),  are  becoming  increasingly  available 
off  the  shelf,  they  still  require  substantial  computation  resources,  usually  available  only  in  garrison 
or  at  the  largest  deployed  facilities.  Because  DSSs  are  intended  to  augment  expert  decision  making, 
the  COA  discovery  or  comparison  should  not  be  performed  remotely  by  users  who  lack  first-hand 
knowledge  of  the  situation  on  the  ground.  The  true  challenge,  therefore,  is  to  provide  decision 
support  in  real  time  so  that  decision  makers  can  use  these  mitigation  models  in  the  theater  of 
operations.  To  enable  such  use  in  the  field,  researchers  must  develop  large-scale  systems  that  do 
not  depend  on  physical  access  to  high-performance  computing  systems. 

The  previous  discussion  also  assumes  that  large-scale  DSSs  can  be  developed  from  various 
component  sociocultural  models  and  that  information  from  the  models  for  understanding, 
detecting,  or  forecasting  behavior  can  be  directly  transferred  to  one  of  the  mitigation  models. 
However,  given  that  many  models  utilize  unique  data  representation  and  output  formats,  creating 
a  complete  system  can  represent  a  serious  undertaking.  Developers  must  conduct  further  research 
in  fields  such  as  multi-modeling,  model  interoperability,  and  data  fusion  to  generate  models 
capable  of  manipulating  the  outputs  of  sociocultural  understanding  or  predictive  models  into 
proper  inputs  for  mitigation  simulations  in  a  standardized  format. 

Further,  it  is  crucial  to  develop  models  transparent  enough  to  foster  trust  by  real-world  users, 
without  requiring  them  to  become  experts  in  computational  modeling.  A  simulation  system  so 
complex  that  only  an  expert  in  computer  science  or  mathematics  can  run  it  will  not  be  effective  for 
decision  support  of  mitigation  policies.  Transparency  can  be  facilitated  through  adequate  training 
on  how  to  apply  these  models  and  interpret  the  results  when  identifying  possible  COAs  and 
analyzing  their  implementation  in  a  particular  sociocultural  environment.  However,  significant 
research  in  human  computer  interaction  is  still  necessary  to  determine  how  to  design  user 
interfaces  for  these  computational  mitigation  models  to  ensure  effective,  intuitive  use,  and 
determine  how  best  to  incorporate  them  into  the  MDMP. 

The  final  problem  that  affects  all  computational  models  described  in  this  book  centers  on  data. 
While  many  of  these  reasoning  and  simulation  approaches  are  designed  to  handle  large  quantities 
of  data,  the  outcomes  will  only  be  useful  for  discovering,  analyzing,  and  comparing  COAs  if  the 
input  data  (or  the  data  used  for  the  underlying  models  for  understanding,  detecting,  and 
forecasting  sociocultural  dynamics)  are  reliable.  For  many  social  analyses,  Internet  social  media 
present  a  new  stream  of  human-generated  data  that  may  serve  as  an  open  source  "sensor" 
providing  insight  into  social  processes.  However,  other  tasks  require  data  extraction  from  widely 
available,  unstructured  textual  sources  (news  media,  public  statements,  etc.)  or  qualitative 
information  (interviews,  surveys,  etc.)  collected  through  scholarly  fieldwork  or  official  human 
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intelligence  channels.  As  Chapter  13  discussed  in  depth,  integrating  this  wealth  of  information  into 
computational  mitigation  models  presents  a  major  challenge. 


7.  Summary 

This  chapter  has  examined  the  potential  for  computational  models  to  aid  decision  makers  in 
developing,  analyzing,  and  comparing  possible  COAs  for  mitigating  sociocultural  factors  in  the 
environment.  Such  tools  have  the  potential  to  significantly  ease  the  analytic  and  decision-making 
burden  faced  by  humans  in  the  field,  allowing  users  to  process  large  quantities  of  complex  social  or 
behavioral  data  that  would  otherwise  be  inaccessible.  Using  large-scale  data  analytics  similar  to 
those  applied  by  companies  such  as  Amazon  or  Google,  these  modeling  frameworks  can  leverage 
the  wealth  of  sociocultural  information  gained  from  understanding,  detecting,  and  forecasting 
aspects  of  the  human  terrain.  Mitigation  models,  therefore,  can  augment  the  existing  MDMP, 
facilitating  the  development  of  robust  COAs  while  also  exploiting  the  expertise  and  knowledge  of 
decision  makers  in  the  field.  However,  many  of  these  enabling  technologies  are  still  in  the  early 
stages  of  research  and  development,  and  system  developers  must  overcome  significant 
technological  and  training  challenges  before  mitigation  models  will  become  widely  operational  for 
real-world  situations. 
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1.  Introduction 

Interactive  data  visualization  can  greatly  assist  decision  makers  in  comparing  and  contrasting  their 
options  as  they  choose  a  course  of  action  for  mitigation.  Admittedly,  ineffective  visualizations  can 
lead  to  poor  decisions  (consider,  for  example,  the  many  descriptions  of  unintelligible  charts  used  in 
the  process  of  deciding  whether  or  not  to  launch  the  space  shuttle  Challenger  (Tufte,  1997)).  By 
contrast,  effective  visualizations  can  lead  to  a  deeper  understanding  of  the  robustness  of  various 
courses  of  action,  resulting  in  better  outcomes  for  a  greater  proportion  of  decisions.  In  this 
chapter,  we  cover  the  underpinnings  of  effective  visualizations  that  help  decision  makers  better 
comprehend  their  options.  Because  using  interactive  visualization  to  compare  and  contrast  courses 
of  action  represents  a  nascent  area  of  study,  we  also  discuss  the  research  to  date  in  this  area  and 
describe  promising  directions  for  future  investigation. 

1.1.  Definition 

In  essence,  visualization  for  mitigation  means  supporting  option  awareness.  In  this  chapter,  we  use 
the  definition  of  option  awareness  provided  by  Drury,  Pfaff,  Klein,  and  Liu  (2013,  p.  658):  "... 
individual  and  team  decision  makers  develop  option  awareness  (OA):  the  perception  and 
comprehension  of  the  relative  desirability  of  the  available  options,  as  well  as  the  underlying 
factors,  trade-offs,  and  tipping-points  that  explain  that  desirability..."  Option  awareness  assists 
users  in  selecting  a  good  course  of  action  by  creating  an  understanding  of  how  effective  that  action 
will  be  under  various  conditions.  This  requires  forecasting  of  plausible  futures,  starting  with  the 
particulars  of  a  specific  current  situation.  Ultimately,  visualizations  that  promote  option  awareness 
should  present  the  landscape  of  plausible  outcomes  for  each  option  in  a  way  that  enables  the  user 
to  easily  compare  those  landscapes  and  choose  the  most  robust  option. 

1.1.1.  Types  of  decisions 

Most  visualization  tools  today  simply  present  situation  awareness  data;  the  developers  presume 
that  the  user  will  extrapolate  from  the  details  of  a  situation  to  decide  on  a  course  of  action.  This 
leaves  all  analysis  of  options  and  forecasting  of  outcomes  to  human  cognition,  with  none  of  the 
burden  carried  by  machines  capable  of  analyzing  many  options  quickly.  Computer-supported 
option  awareness  alleviates  this  overload  by  presenting  visualizations  to  augment  cognition  and 
increase  understanding. 


1  Approved  for  Public  Release;  Distribution  Unlimited.  Case  Number  14-2487 
Copyright  ©  2014  The  MITRE  Corporation. 
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As  with  situation  awareness,  individuals  can  achieve  progressively  higher  levels  of  option 
awareness,  and  new  interactive  visualization  tools  can  give  decision  makers  much  broader  and 
deeper  views  of  their  options.  Figure  1  shows  the  three  levels  of  option  awareness,  with  each  level 
building  on  the  insight  and  understanding  gained  from  the  previous  level.  A  visualization  that 
enables  option  awareness  level  1  enables  users  to  see  and  understand  the  comparative  robustness 
of  their  options  more  rapidly.  If  the  visualization  enables  option  awareness  level  2,  users 
understand  the  key  trade-offs  between  those  options  and  the  factors  that  make  one  option  more 
robust  than  another.  Option  awareness  level  3,  which  builds  on  levels  1  and  2,  enables  users  to 
propose  more  effective  options  based  on  a  deep  understanding  of  relative  robustness  and  the 
factors  driving  that  robustness. 


fa 


Perception  and 
comprehension  of 
relative  robustness 
of  alternative 
options 

Level  1 


Perception  and 
comprehension  of 
relationships 
between  factors 
underlying  the 
option  outcomes 

Level  2 


Projection  of 
underlying 
relationships  to 
adjusted  or  new 
options 

Level  3 


-►  Decision 


Figure  1.  Levels  of  option  awareness.  Adapted  with  permission  from  Klein,  Drury,  Pfaff,  &  More 
(2010). 


1.1.2.  Types  of  data 

A  key  driver  for  visualization  in  general  and  option  awareness  visualization  in  particular  is  the  type 
of  data  to  be  visualized.  Fundamentally,  the  option  awareness  data  includes: 

•  A  set  of  alternative  options 

•  A  set  of  conditions  that  can  vary  or  are  unknown  (uncertainty) 

•  Criteria  for  evaluating  the  options 

•  Evaluation  of  the  options 

The  evaluations  can  be  generated  by  computer  simulation  or  by  a  group  of  subject  matter  experts, 
or  can  be  found  from  historical  data.  They  result  from  examining  each  option  across  a  range  of 
possible  conditions  that  could  vary  or  may  be  unknown  and  measuring  the  outcome  in  terms  of 
each  relevant  criterion  (e.g.,  cost  or  lives  lost).  The  complexity  in  option  awareness  data  stems 
from: 


•  Options  that  are  multi-faceted  (e.g.,  the  option  has  two  aspects  that  may  interact  with  each 
other  in  some  way) 

•  Varying  or  unknown  conditions,  including  aspects  of  the  current  situation  (e.g.,  a  flu 
outbreak  with  unknown  lethality)  or  uncertainty  about  option  effectiveness  (e.g.,  what 
percentage  of  people  will  comply  with  a  request  to  stay  home  from  work) 
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Table  1  shows  this  breakdown  for  the  simple  notional  dataset  of  the  virus  outbreak  that  we  use  as 
an  illustration  throughout  this  chapter.  In  this  example,  the  decision  maker  must  determine  how  to 
respond  to  the  outbreak.  The  option  has  two  components:  (a)  whether  to  implement  social 
distancing  (i.e.,  ask  schools  and  workplaces  to  close)  and  (b)  what  percentage  of  the  population  to 
vaccinate  (10%  vs.  50%).  These  factors  interact  with  each  other  because  social  distancing  would 
result  in  fewer  people  becoming  exposed  to  the  virus,  and  thus  vaccination  would  have  less  impact 
on  the  outcome.  The  uncertainty  in  this  case  comes  from  not  knowing  the  deadliness  of  the  new 
virus.  Here  we  consider  only  three  possibilities:  30%,  60%,  or  90%  of  people  exposed  and  not 
vaccinated  will  die  from  the  virus.  A  real  scenario  would  probably  involve  multiple  unknown 
variables  (e.g.,  unknown  lethality  +  unknown  percentage  of  the  public  that  would  comply  with 
social  distancing)  and  would  require  evaluation  of  tens  to  hundreds  of  possibilities  (not  just  three). 
This  would  further  complicate  the  data,  resulting  in  an  even  greater  need  for  effective  visualization 
to  aid  the  decision  maker  in  understanding  the  robustness  of  each  option  and  the  underlying 
factors.  In  the  illustrative  example  discussed  in  this  chapter,  the  criteria  we  use  to  assess  the 
goodness  of  each  option  are  cost  and  lives  lost. 
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Table  1.  Notional  option  awareness  data:  Most  aggregation  (top  left)  to  most  detailed  (bottom 
right) 


Criteria 

Option 

Single  Weighted  Score  (i.e.,  Score) 

Each  Criterion  Separately  (i.e.,  Cost,  Lives) 

Option  Deadliness  Score 

Option  Deadliness  Cost  Lives 

OptionNIO  <not  shown  (1,  33,  65} 

OptionNIO  <not  shown>  ($34,  $58,  $82}  (243,  486,  729} 

Single  Name  (i.e., 
Option) 

OptionN50  <not  shown>  (35,  53,  71} 

OptionN50  <not  shown>  ($63,  $77,  $90}  (135,  270,  405} 

Combined  to  Show 

OptionYlO  <not  shown>  (81,  85,  88} 

OptionYlO  <not  shown>  ($37,  $40,  $43}  (27,  54,  81} 

Robustness 

OptionY50  <notshown>  {77,79,81} 

OptionY50  <not  shown>  ($76,  $78,  $79}  (15,  30,  45} 

(i.e.,  criteria  includes  set  of 

values,  one  from  each 
possible  Deadliness 

Dist  Vacc  Deadliness  Score 

Dist  Vacc  Deadliness  Cost  Lives 

condition) 

Each  Aspect 

No  10  <notshown>  {1,33,65} 

No  10  <not  shown>  ($34,  $58,  $82}  (243,  486,  729} 

Separately 

No  50  <not  shown>  {35,53,71} 

No  50  <not  shown>  ($63,  $77,  $90}  (135,  270,  405} 

(i.e.,  Dist,  Vacc) 

Yes  10  <not  shown>  (81,  85,  88} 

Yes  10  <not  shown>  ($37,  $40,  $43}  (27,  54,  81} 

Yes  50  <not  shown>  (77,  79,  81} 

Yes  50  <not  shown>  ($76,  $78,  $79}  (15,  30,  45} 

Option  Deadliness  Score 

Option  Deadliness  Cost  Lives 

OptionNIO  30 

65 

OptionNIO  30 

$34k  243 

OptionNIO  60 

33 

OptionNIO  60 

$58k  486 

OptionNIO  90 

1 

OptionNIO  90 

$82k  729 

OptionN50  30 

71 

OptionN50  30 

$63k  135 

OptionN50  60 

53 

OptionN50  60 

$77k  270 

Single  Name 

OptionN50  90 

35 

OptionN50  90 

$90k  405 

(i.e.,  Option) 

Uncertainty 

OptionYlO  30 

88 

OptionYlO  30 

$37k  27 

OptionYlO  60 

85 

OptionYlO  60 

$40k  54 

OptionYlO  90 

81 

OptionYlO  90 

$43k  81 

OptionY50  30 

81 

OptionY50  30 

$76k  15 

OptionY50  60 

79 

OptionY50  60 

$78k  30 

Each  Possibility  Separately 

(i.e.,  Deadliness  =  30,  60,  or 

OptionY50  90 

77 

OptionY50  90 

$79k  45 

90) 

Dist  Vacc  Deadliness  Score 

Dist  Vacc  Deadliness  Cost  Lives 

No  10  30 

65 

No  10  30 

$34k  243 

No  10  60 

33 

No  10  60 

$58k  486 

No  10  90 

1 

No  10  90 

$82k  729 

No  50  30 

71 

No  50  30 

$63k  135 

Each  Aspect 

No  50  60 

53 

No  50  60 

$77k  270 

Separately 

No  50  90 

35 

No  50  90 

$90k  405 

(i.e.,  Dist,  Vacc) 

Yes  10  30 

88 

Yes  10  30 

$37k  27 

Yes  10  60 

85 

Yes  10  60 

$40k  54 

Yes  10  90 

81 

Yes  10  90 

$43k  81 

Yes  50  30 

81 

Yes  50  30 

$76k  15 

Yes  50  60 

79 

Yes  50  60 

$78k  30 

Yes  50  90 

77 

Yes  50  90 

$79k  45 
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We  follow  visualization  expert  Ben  Shneiderman's  visual  information  seeking  mantra  of  "overview 
first,  zoom  and  filter,  then  details  on  demand"  (Shneiderman,  1996,  p.336)  to  make  visualizations 
of  the  data  more  manageable  for  a  decision  maker.  At  the  highest  level,  the  data  presented  for 
each  option  includes  the  name  of  the  option  and  a  set  of  scores  for  each  option.  The  top  left  cell  in 
Table  1  shows  the  highest  level  of  aggregation  of  the  data:  an  overview.  The  two  aspects  of  the 
option  (dist  and  vacc)  are  combined  into  a  name  that  describes  the  complete  option  (e.g., 
OptionNIO).  The  two  criteria  (lives  and  cost)  are  combined  into  a  single  weighted  score.  The 
specific  conditions  under  which  each  option  was  tested  (i.e.,  the  deadliness  percentages  of  30,  60, 
and  90)  have  been  grouped  by  option,  resulting  in  a  set  of  values  that  represent  the  plausible 
outcomes  for  that  option  under  uncertainty  (the  uncertainty  about  the  deadliness  of  the  virus).  At 
the  other  end  of  the  spectrum,  as  decision  makers  drill  down  to  obtain  more  details,  they  would 
move  toward  the  bottom  right  cell  in  the  table,  which  shows  the  highest  level  of  detail  for  this 
simple  example  dataset. 

The  characteristics  of  the  data  constrain  the  space  of  relevant  visualization  techniques.  Users  can 
attain  option  awareness  level  1— understanding  the  robustness  of  each  course  of  action— even 
from  the  most  aggregated  form  of  the  data,  the  overview,  if  visualized  effectively.  However, 
displaying  only  the  aggregate  data  hides  the  underlying  factors  that  drive  robustness.  To  achieve 
level  2  option  awareness  users  require  a  capability  to  interact  with  the  data,  filter  the  data  to  show 
only  the  most  worthwhile  options,  and  drill  down  into  the  data  to  find  more  details  so  they  can 
begin  to  understand  the  factors  driving  the  outcomes.  This  helps  users  to  understand  the  trade¬ 
offs:  for  example,  two  options  may  have  a  relatively  similar  average  outcome,  but  one  has  a  much 
better  best  case  and  much  worse  worst  case  than  the  other. 

1.2.  Why  Visualization  Matters 

The  visual  interface  constitutes  the  last  gateway  between  data  about  the  robustness  of  the  options 
and  the  decision  maker's  mind.  A  clear  presentation  of  the  option  awareness  data  reduces  the 
cognitive  burden.  However,  if  the  chosen  visualization  technique  obfuscates  options  and  outcomes, 
decision  makers  will  find  it  difficult  to  fully  comprehend  the  resulting  analysis  of  their  options. 

Designers  must  create  cognitively  and  perceptually  effective  visualizations  that  allow  the  user  to 
interact  with  the  visualization  in  appropriate  and  meaningful  ways.  Something  as  simple  as  the 
terminology  used  to  convey  the  data  can  affect  risk  perception  and  the  resulting  decisions  (e.g., 
due  to  the  framing  effect)  (Ancker,  Senathirajah,  Kukafka,  &  Starren,  2006).  Further,  if  users  do  not 
understand  the  inner  workings  of  a  tool,  particularly  a  modeling  and  simulation  tool,  they  may 
blindly  trust  the  tool  too  much  or  lose  trust  in  the  tool  altogether  (Sheridan,  2002).  To  realize  the 
powerful  benefits  of  using  interaction  visualizations  to  analyze  options,  new  visualizations  and 
visual  analytics  tools  must  effectively  address  these  types  of  cognitive  issues. 

1.3.  What  Makes  Visualization  Difficult 

Option  awareness  visualization  comes  with  its  own  unique  set  of  challenges  that  future  research 
must  address.  At  a  high  level,  some  of  the  challenges  in  this  space  include: 

•  Complexity  of  the  data 
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o  Achieving  greater  levels  of  option  awareness  will  require  visualizations  that  provide 
deeper  understanding  of  the  underlying,  inherently  complex  data.  How  do  we 
effectively  portray  the  robustness  of  each  option?  How  do  we  create  visualizations 
that  clearly  show  users  how  the  different  factors  interact? 

•  Diverse  users  without  statistics  backgrounds 

o  As  this  chapter  will  show,  statistical  analysis  and  concepts  such  as  mean  and  standard 
deviation  are  often  used  to  summarize  the  set  of  scores  that  represent  the  robustness 
of  an  option  (e.g.,  in  the  example  above,  OptionNIO  has  a  set  of  scores  {1,  33,  65}). 
Yet  many  decision  makers  do  not  have  a  statistics  background.  How  do  we  create 
interactive  visual  analytics  tools  useful  to  groups  of  stakeholders  who  have  diverse 
backgrounds  and  experiences?  Which  information  could  and  should  the  visualization 
abstract  away? 

•  Need  for  effective  visualization  design  principles  to  enable  trustworthy,  defensible  decision 

support 

o  Much  has  been  written  about  creating  effective  situation  awareness  displays,  but 
little  about  creating  effective  option  awareness  displays.  Rigorous  design  principles 
will  lead  to  visualizations  that  effectively  support  option  awareness.  Further,  in  the 
context  of  selecting  between  options,  good  design  principles  are  important  not  just 
for  cognitive  and  perceptual  reasons,  but  also  for: 

■  Ensuring  appropriate  levels  of  trust  -  While  decision  makers  will  not  rely 
exclusively  on  a  visual  analytics  tool  for  information,  that  tool  will  be  one  of 
several  sources.  Decision  makers  need  a  way  to  determine  the  reliability  of  the 
results  and  the  level  of  trust  to  put  in  what  they  are  seeing. 

■  Ensuring  ethical  designs  -  Designers  must  create  visualizations  that  show  the 
data  in  a  way  that  does  not  unduly  influence  the  user's  decisions. 

■  Ensuring  defensible  results  -  Option  awareness  level  2  gives  decision  makers  the 
understanding  they  need  to  explain  or  defend  the  reasons  why  they  chose  one 
option  over  another. 

The  remainder  of  this  chapter  presents  more  details  on  these  challenges. 

1.5.  How  Visualization  Fits  into  the  Bigger  Picture 

This  chapter  centers  on  visualization  for  comparing  options  and  gaining  understanding  of  the  trade¬ 
offs,  but  comparing  alternative  options  represents  only  one  step  in  the  mitigation  process.  In 
reality,  visualization  can  help  in  other  areas  related  to  mitigation;  for  instance,  in  generating 
alternative  options,  assisting  those  tasked  with  executing  the  selected  option,  or  monitoring  the 
effectiveness  of  the  selected  option. 

First,  a  subject  matter  expert  needs  to  identify  and  generate  alternative  options  -  whether  proven 
or  novel  -  before  they  can  be  modeled  and  compared.  Most  real-world  cases  fall  into  the  middle  of 
this  continuum.  Well-known  options  typically  involve  checklists  and  well-documented  processes. 
Visualization  can  allow  users  to  compare  all  of  the  options  and  can  create  the  basis  for  generating 
new  variations.  By  observing  which  factors  lead  to  good  or  bad  outcomes,  a  user  acquires  the  basis 
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for  creating  new,  even  more  effective  options.  Visualizations  that  provide  option  awareness  levels 
2  and  3  assist  with  this  process  (see  sections  3.1.2  and  3.1.3  in  this  chapter). 

Second,  once  decision  makers  select  a  particular  action  they  must  prescribe  how  to  execute  the 
action.  The  individuals  responsible  for  carrying  out  the  action  will  need  detailed  instructions.  The 
details  must  include  what  to  do  and  how  to  do  it,  and  may  require  discussion  and  revision  (e.g.,  the 
Postal  Service  may  be  asked  to  deliver  vaccines  to  homes).  Those  responsible  for  carrying  out  the 
action  must  also  know  what  resources  will  be  available  to  support  the  action  (e.g.,  how  much 
vaccine  to  deliver)  and  the  scope  of  the  action  (e.g.,  to  whom  to  deliver  the  vaccines).  Visualization 
could  help  decision  makers  communicate  this  type  of  information  to  the  action  takers. 

Third,  as  the  actions  take  place  and  the  situation  evolves,  decision  makers  must  assess  the  results 
and  adapt  the  response  based  on  that  assessment.  For  example,  if  people  are  asked  to  stay  home 
from  work,  someone  must  determine  what  percentage  of  the  public  is  complying.  The  greater  the 
percentage  of  people  who  comply  with  what  is  asked,  the  more  effective  the  option.  This  calls  for 
practices  and  tools  that  enable  the  initial  data  and  the  reasons  for  selecting  a  particular  option  to 
be  available  when  assessing  the  effectiveness  of  the  actions  being  taken.  This  would  contribute  to 
effective  mitigation  because  decision  makers  could  better  understand  how  well  the  situation  on 
the  ground  matched  the  predictions  of  the  original  models  used  to  choose  among  options.  All  of 
these  areas  call  for  research,  including  work  to  build  a  system  that  provides  early  indicators  of  how 
well  the  mitigation  actions  are  working. 

We  limit  the  scope  of  this  chapter  to  visualization  for  comparing  the  relative  robustness  of  a  set  of 
options,  understanding  the  trade-offs  between  those  options,  and  gaining  a  deep  enough 
understanding  of  those  factors  to  generate  novel  options.  The  assessment  phase  represents  a  full 
circle  back  to  using  visualization  for  understanding  and  detection.  In  the  remainder  of  this  chapter, 
we  first  briefly  describe  how  decision  makers  use  option  analysis  today,  which  leads  into  the 
discussion  of  the  state  of  the  art.  We  then  discuss  areas  that  require  more  research  to  further 
advance  the  state  of  interactive  visualization  tools  for  option  awareness. 


2.  State  of  the  Practice 

This  section  summarizes  the  primary  techniques  used  for  visualization  today,  highlights  some 
findings,  and  suggests  shortfalls  in  current  capabilities. 

2.1.  Manual  Techniques 

Today,  most  analysts  compare  different  options  by  hand,  perhaps  with  the  assistance  of  a 
spreadsheet  to  capture  the  data,  and  then  visualize  the  results  in  a  simple  static  diagram  that  can 
be  placed  on  a  presentation  slide.  For  example,  in  its  Analysis  of  Alternatives  (AoA)  Handbook 
(AFMC,  2008)  the  Air  Force  includes  a  section  on  presenting  the  results  of  the  AoA,  as  illustrated  in 
Figure  2.  This  type  of  chart  is  sometimes  referred  to  as  a  horse  blanket  because  of  its  colorful 
design. 
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Figure  2.  Horse  blanket  -  static  visualization  for  comparing  options  (AFMC,  2008). 


The  AoA  Handbook  uses  this  horse  blanket  (or  heatmap)  visualization  approach  to  compare  options 
("Alternative  1,"  etc.)  across  a  number  of  different  conditions  ("MT  1"  is  Mission  Task  1),  for  a 
number  of  different  criteria  ("MoE  1-1"  stands  for  Measure  of  Effectiveness,  a  measure  relevant  to 
the  first  mission  task).  The  handbook  stresses  that  designers  must  apply  a  method  for  mapping 
colors  based  on  the  score  relative  to  a  meaningful  threshold  and  discourages  use  of  weighted 
averages  across  criteria,  as  these  can  be  misleading.  If  we  apply  this  static  visualization  to  our 
example  dataset  shown  in  Table  1  we  obtain  the  visualization  in  Figure  3.  Each  cell  shows  the 
average  value. 


Option  N 10 
Option  l\I5Q 
OptionYlO 
OptionY50 


Cost  Lives 


Figure  3.  Horse  blanket  (heat  map)  applied  to  notional  virus  outbreak  dataset. 


The  simple  example  in  Figure  3  reveals  the  drawbacks  of  this  visualization  method.  By  showing  only 
the  average  values,  it  does  not  indicate  robustness  across  plausible  deadliness  percentages  (e.g., 
best  and  worst  case  values  for  each  option).  This  means  it  does  not  provide  option  awareness  level 
1.  Since  it  does  not  show  the  robustness,  it  also  cannot  show  the  factors  underlying  that 
robustness.  Additionally,  users  have  no  easy  way  to  interact  with,  filter,  or  drill  down  into  the  data 
to  explore  the  relationships  among  factors  and  thus  to  achieve  option  awareness  level  2.  Section 
2.2  briefly  discusses  a  few  emerging  visualization  techniques  used  to  achieve  option  awareness. 

2.2.  Interactive  Techniques 

In  addition  to  the  static  visualizations  found  in  presentations,  there  are  commercial  software 
products  that  provide  interactive  visualizations  for  comparing  options.  These  interactive  tools  tend 
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to  depend  heavily  on  different  analytic  techniques.  Some  of  these  tools  provide  some  support  for 
option  awareness  level  2,  in  which  a  person  understands  the  factors  underlying  the  outcomes. 
Sensitivity  analysis  and  classification  trees  are  two  examples  of  techniques  found  in  these  products. 

Sensitivity  analysis  allows  users  to  determine  how  much  of  the  variance  in  the  outcome  is  due  to 
different  sources  of  uncertainty.  For  example,  based  on  the  data  in  Table  1,  the  variance  in  the  set 
of  scores  associated  with  each  option  results  from  the  choice  of  whether  to  implement  social 
distancing,  the  choice  of  how  much  of  the  population  to  vaccinate,  and  the  deadliness  of  the  virus. 

The  simplest  approach  to  sensitivity  analysis  involves  varying  a  single  condition  at  a  time  and 
examining  the  impact  on  the  outcome.  Tornado  charts  show  the  results  of  this  approach.  For 
example,  in  the  tornado  chart  in  Figure  4,  the  occupancy  rate  of  schedule  flights  (i.e.,  capacity  of 
scheduled  flights)  was  varied  from  40%  (worst  case)  to  60%  (best  case).  The  chart  shows  that  if 
occupancy  is  at  only  40%  the  airline  will  lose  money;  however,  if  occupancy  hits  60%  the  airline  will 
profit  by  approximately  $35,000  annually.  The  sort  order  places  the  variable  that  causes  the  most 
variance  in  the  outcome  on  top.  In  this  example,  the  occupancy  of  scheduled  flights  has  the 
greatest  influence  on  the  annual  profit  while  the  cost  of  insurance  has  the  least. 

Senslt  Sensitivity  Analysis 
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Figure  4.  Tornado  chart.  Adapted  with  permission  from  Sensit  (2013). 


Sensitivity  analysis  shown  in  tornado  charts  falls  short  on  option  awareness  level  1.  Since  each 
factor  is  varied  independently,  the  analysis  involves  a  single  notional  scenario  rather  than  values 
from  multiple  varying  or  uncertain  conditions.  However,  such  charts  do  start  to  provide  option 
awareness  level  2  by  showing  the  factors  that  have  the  greatest  impact  on  the  outcome.  For 
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example,  if  we  consider  our  notional  example  from  Table  1,  our  tornado  chart  would  look  like 
Figure  5  for  a  worst  case  scenario.  The  effect  of  each  factor  (Dist,  Vacc,  Deadliness)  on  the  total 
number  of  lives  lost  clearly  identifies  social  distancing  as  the  most  important  factor  because  it 
changes  the  percentage  of  the  population  exposed  to  the  virus.  It  does  not  show  interaction  effects 
(i.e.,  that  if  social  distancing  is  implemented  the  percentage  of  the  population  vaccinated  matters 
less),  but  does  provide  a  simple  method  of  ranking  the  importance  of  the  underlying  factors.  The 
need  remains  for  specific  visualization  approaches  that  could  begin  to  explain  conditions  under 
which  a  particular  option  will  produce  a  better  or  worse  outcome. 


Social  Distancing 

Deadliness  of  Virus 

Population  Vaccinated 


Lives  Lost 

Figure  5.  Tornado  chart  showing  notional  virus  outbreak  dataset. 


A  full  summary  of  all  the  commercial  solutions  is  beyond  the  scope  of  this  chapter.  The  example  in 
Figure  4  came  from  Senslt,  an  add-on  for  Excel.  Other  companies,  such  as  Vanguard  Software,  have 
similar  software  solutions  for  generating  non-interactive  visualizations.2  We  mention  them  purely 
as  examples  of  existing  commercial  software  that  represents  the  types  of  visual  analytics  tools  that 
will  eventually  support  users  in  achieving  progressively  higher  levels  of  option  awareness. 

Another  interactive  visualization  technique  that  aids  in  reaching  option  awareness  level  2  is  a 
decision  tree.  The  data  mining  algorithms  that  produce  classification  trees  can  generate  decision 
trees  that  bin  the  outcomes  into  categorical  groups  (e.g.,  good,  moderate,  and  poor  outcomes). 
The  leaf  nodes  in  a  tree  visualization  show  the  number  of  good,  moderate,  or  poor  outcomes  that 
resulted  from  following  a  particular  path  down  the  tree.  Figure  6  gives  an  example  for  the  notional 
virus  dataset,  showing  that  the  decision  to  implement  social  distancing  was  the  most  crucial  factor 
in  ensuring  good  outcomes.  Social  distancing  led  to  the  fewest  number  of  lives  lost,  regardless  of 
the  percentage  of  the  population  vaccinated  or  the  deadliness  of  the  virus. 


2  See  http://www.vanguardsw.com/products/vanguard-system/risk-analysis.htm 
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Figure  6.  Decision  tree  showing  option  awareness  level  2  for  notional  dataset's  number  of  lives  lost. 


The  factors  that  have  the  most  influence  on  the  outcome  appear  at  the  top  of  the  tree.  This 
approach  helps  explain  the  factors  behind  a  particular  outcome.  As  in  the  case  with  the  manual 
techniques,  much  work  remains  to  support  option  awareness  effectively. 

2.3.  Lessons  Learned  from  Practice 

Readers  must  keep  in  mind  that  while  this  chapter  uses  a  small,  simple,  notional  dataset  for 
illustrative  purposes,  in  practice  the  source  dataset  can  be  large  and  complex.  The  scalability  of  the 
visualization  techniques  and  interaction  with  the  visualizations  must  enable  display  of  large 
complex  datasets  in  ways  that  domain  experts  can  quickly  understand.  Often  the  complexity 
involved  means  that  people  resort  to  very  simplistic  stoplight  chart  indicators  (as  in  the  horse 
blanket),  consider  only  a  single  scenario  (as  in  the  tornado  chart),  or  show  only  the  best  and  worst 
cases  for  a  given  option. 

2.4.  Gaps 

Better  interactive  visualization  tools,  especially  those  that  appeal  to  domain  experts,  are  needed  to 
bring  these  capabilities  to  an  audience  without  a  strong  background  in  statistics.  Users  need  tools 
that  can  enable  all  three  option  awareness  levels  and  provide  a  seamless  transition  between  levels. 
These  tools  must  be  accessible  for  end  users,  with  appropriate  training  on  basic  statistics  and 
advanced  visualization  techniques  where  necessary.  Research  is  just  beginning  to  address  some  of 
these  issues. 

3.  Visualization  Techniques  for  Option  Awareness 

While  existing  visualization  techniques  can  show  the  necessary  types  of  data,  using  these 
techniques  to  visualize  option  awareness  data  represents  a  new  direction.  In  this  section,  we 
describe  a  few  techniques  that  developers  could  incorporate  in  future  tools  to  promote  option 
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awareness  levels  1  and  2.  How  well  visualization  can  enable  and  support  option  awareness  level  3 
remains  an  open  question. 


3.1.  Option  Awareness  Levels 


3.1.1.  Option  awareness  level  1  -  Showing  distribution 

Option  awareness  level  1  requires  tools  that  show  the  relative  robustness  of  each  option.  This 
means  displaying  the  set  of  plausible  outcomes  for  each  option.  The  challenge  for  option 
awareness  level  1  lies  in  showing  the  distributions  of  outcomes.  Designers  and  developers  must 
determine  how  much  statistical  analysis  is  needed  and  appropriate  for  the  audience  and  what  can 
be  simplified. 


Visualization  Approach 

Box  plots,  also  known  as  box-and-whisker  plots,  are  a  statistical  visualization  technique  created  by 
Tukey  (1977).  There  are  many  variations  of  the  box  plot;  a  basic  plot  is  shown  in  Figure  7. 
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Figure  7.  Box  plot  showing  distribution  of  plausible  futures  (option  awareness  level  1)  for  four 
options. 


Data 

Each  of  the  four  notional  options  (Options  A,  B,  C,  and  D)  in  the  dataset  had  35-40  possible 
outcomes  in  the  dataset  represented  in  Figure  7.  The  algorithms  calculated  the  minimum  cost 
possibility,  maximum  cost  possibility,  first  quartile,  and  third  quartile. 

Encoding 

The  vertical  line  represents  the  range  from  the  best  case  to  worst  case.  The  box  in  the  middle 
shows  the  middle  50%  of  all  plausible  outcomes  for  that  option  (i.e.,  the  25th-75th  percentile). 
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Interaction 

Users  may  wish  to  sort  data  based  on  the  compactness  of  the  range  of  plausible  outcomes  (length 
of  the  line),  the  best  outcome  (bottom  of  the  line),  the  worst  outcome  (top  of  the  line),  or  the 
characteristics  of  the  middle  50%  of  plausible  outcomes  (size  of  the  box,  top  or  bottom  of  the  box). 

Advantages  and  Disadvantages 

The  box  plot  provides  an  excellent  statistical  summary  of  the  data,  allowing  users  to  compare  best, 
worst,  or  most  likely  outcomes  under  uncertainty.  However,  users  must  understand  the  concepts 
of  the  25th  and  75th  percentiles  and  basic  statistics  to  interpret  the  plot. 

Alternative  Visualizations 

There  are  many  alternative  visualization  techniques  that  show  the  actual  values  rather  than  a 
statistical  summary  of  distribution,  as  well  as  many  variations  on  the  basic  box  plot.  The  most  basic 
alternative  technique  is  the  histogram.  Figure  8  shows  the  landscape  of  plausible  outcomes  and 
provides  option  awareness  level  1.  The  challenge  in  developing  such  a  histogram  is  to  appropriately 
bin  the  data  into  groups  (e.g.,  $4,000-$4,999  is  a  bin).  A  wise  selection  of  bins  can  create  an 
effective  way  to  show  option  awareness  level  1  using  the  actual  data  points  rather  than  statistical 
summary  data. 


Figure  8.  Histogram  showing  landscape  of  plausible  futures. 


Variations  of  the  box  plot  show  values  in  addition  to  the  statistical  summary  data.  These  variations 
include  density  plots,  density  plots  with  a  rug,  bean  plots,  and  violin  plots;  Yau  (2013)  gives  a  brief 
tutorial  on  these  plots  and  how  to  implement  them  using  the  programming  language  R.  Bean  plots 
show  all  of  the  specific  data  points  while  addressing  the  issue  of  overlapping  points.  Other 


420  |  Interactive  data  visualization 


proposed  techniques  include  showing  all  outcomes  as  lines,  but  making  each  line  semi-transparent 
to  reveal  where  many  points  are  superimposed  on  each  other  (Barnes,  2013). 

Some  past  work  has  compared  these  visualization  designs  and  discussed  the  trade-offs  among 
different  visualization  techniques  for  showing  the  distribution  of  values  (Ibrekk  &  Morgan,  1987). 
The  main  takeaways  were  that  simple  explanations  of  the  visualizations  aided  understanding,  but 
only  minimally,  and  basic  visualization  techniques  like  box  plots  or  the  semi-transparent  approach 
offered  particular  promise.  Visualization  techniques  for  showing  distribution  of  values  have  seen 
little  adoption,  perhaps  because  of  the  need  for  users  to  understand  basic  statistics.  Future 
research  should  address  the  comparative  effectiveness  of  these  techniques  to  support  option 
awareness  and  gain  adoption. 

While  all  the  examples  provided  may  lead  to  option  awareness  level  1  because  they  show  the  data 
distribution,  they  do  not  provide  insight  into  the  underlying  factors  that  caused  the  difference  in 
outcomes.  That  insight  requires  effective  multidimensional  visualization  techniques  that  show  the 
relative  importance  of  factors  that  result  in  good  or  bad  outcomes  for  a  particular  option. 

3.1.2.  Option  awareness  level  2  -  Showing  multidimensional  data 

Option  awareness  level  2  requires  a  way  to  display  the  underlying  factors  driving  the  outcomes  and 
the  interactions  among  those  factors.  In  our  simple  notional  dataset,  this  means  we  need  a  way  to 
effectively  understand  the  impact  that  social  distancing,  vaccinations,  and  deadliness  of  the  virus 
have  on  the  outcomes.  We  also  need  to  know  how  those  variables  interact.  Additional  benefits  of 
visualizing  the  factors  include  that  showing  users  what  factors  affect  better  vs.  worse  outcomes 
provides  a  foundation  for  developing  new  courses  of  action  (see  section  3.1.3).  Furthermore,  users 
can  immediately  see  the  sources  of  uncertainty  and  can  then  question  the  underlying  assumptions 
made  in  the  models.  Moreover,  seeing  the  factors  can  help  users  compare  the  relative  importance 
of  each  source  of  uncertainty  in  a  given  situation.  The  challenge  for  option  awareness  level  2  lies  in 
showing  the  most  critical  factors  and  the  relationships  among  them.  In  many  datasets  this 
challenge  is  further  compounded  by  a  large  number  of  variables. 

Visualization  Approach 

A  key  visualization  technique  in  this  area  is  the  decision  tree,  classification  tree,  or  variant  of  it  (see 
section  2.2  and  Figure  6). 

Data 

The  data  includes  all  of  the  plausible  future  paths  for  all  of  the  options  and  the  corresponding 
outcomes. 

Encoding 

Some  of  the  nodes  in  the  tree  represent  a  decision  point  (e.g.,  whether  to  implement  social 
distancing)  or  variable  (e.g.,  deadliness  of  the  virus).  The  decision  points  or  variables  with  the 
greatest  impact  on  the  outcome  appear  higher  in  the  tree.  The  leaf  nodes  at  the  lowest  level  of  the 
tree  structure  represent  the  outcomes.  Nodes  can  be  combined  and  aggregated  in  cases  where  a 
variable  has  no  influence  on  the  outcome.  For  example,  if  social  distancing  is  implemented  then  the 
outcome  will  always  be  good,  regardless  of  vaccinations  or  deadliness,  so  the  nodes  under  yes  to 
social  distancing  can  be  collapsed  into  a  single  node  that  represents  good  outcomes. 
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Interaction 

Users  may  wish  to  filter  the  data  so  that  only  paths  that  lead  to  good  outcomes  are  shown  in  the 
tree.  They  also  may  wish  to  prune  the  tree  by  selecting  a  value  for  a  particular  option  (e.g.,  no  to 
social  distancing). 

Advantages  and  Disadvantages 

While  decision  trees  can  prove  highly  effective  in  aiding  option  awareness  level  2,  they  can  become 
complex  when  a  situation  involves  many  variables.  Users  may  require  training  to  become  familiar 
with  the  underlying  analysis  and  understand  what  the  tree  shows.  To  attain  option  awareness  level 
2,  users  need  new,  easy-to-grasp  visualizations  that  show  the  interactions  among  factors  and  the 
impact  they  have  on  the  outcomes  (i.e.,  that  display  the  data  from  all  plausible  futures,  not  just 
one). 

Alternative  Visualizations 

There  are  alternative  visualization  techniques,  techniques  other  than  decision  trees,  which  can 
assist  with  option  awareness  level  2.  These  techniques  for  visualizing  multidimensional  data  include 
a  scatterplot  matrix  and  parallel  coordinates.  A  full  discussion  of  approaches  for  multidimensional 
data  visualization,  which  would  help  in  establishing  option  awareness  level  2,  is  outside  the  scope 
of  this  chapter.  We  cover  this  topic  briefly  only  to  help  the  reader  understand  what  research  has 
already  accomplished  and  where  open  research  challenges  remain. 

3.1.3.  Option  awareness  level  3  -  Identifying  new  options 

Once  users  have  option  awareness  levels  1  and  2  they  will  understand  the  factors  underlying  the 
outcomes  for  a  particular  option(s)  and  will  need  a  way  to  create  a  new  option  and  confirm  the 
predicted  robustness  of  that  new  option.  Creating  a  new  option  calls  for  a  capability  to  specify 
options  not  previously  considered  (e.g.,  apply  quality  assurance  to  a  vaccination  program  to  ensure 
a  minimum  vaccination  level).  Interaction  controls  could  help  achieve  option  awareness  level  3 
provided  they  feed  new  option  information  back  to  level  1  or  2  visualizations. 

3.2.  Gaps 

While  visualization  and  interaction  techniques  can  move  us  one  step  beyond  the  current  state  of 
the  practice,  the  biggest  gains  would  come  from  (a)  new  visualization  techniques  that  provide 
option  awareness  level  3,  (b)  visualizations  for  option  awareness  that  are  easy  for  non-statisticians 
to  understand  and  use,  (c)  tests  of  the  effectiveness  of  these  new  techniques  for  supporting 
selection  of  options  in  the  real  world  (e.g.,  which  multidimensional  visualization  techniques  best 
support  selection  of  an  option  and  aid  understanding  of  factors  underlying  the  option's 
robustness),  and  (d)  an  integrated  capability  for  achieving  progressively  higher  option  awareness 
levels  (e.g.,  bringing  together  option  awareness  levels  1,  2,  and  3).  Further  research  is  needed  to 
understand  how  well  existing  techniques  work  when  used  in  real-world  situations  where  the 
uncertainty  would  decrease  over  time  and  the  option  awareness  data  could  be  refined  in  real  time. 
This  calls  for  visualization  methods  that  clearly  show  users  where,  how,  and  why  the  robustness  of 
options  has  changed. 
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4.  State  of  the  Art 

In  this  section  we  focus  on  the  most  recent  advances  and  highlight  cutting-edge  research  on 
visualization  for  option  awareness.  Because  these  techniques  are  new,  much  remains  to  be  done  to 
confirm  their  effectiveness.  In  the  coming  years,  however,  these  capabilities  should  make  their  way 
into  the  operational  community. 

4.1.  Areas  of  Active  Research 

While  researchers  have  produced  many  systems  over  the  years  to  increase  situation  awareness, 
the  distinction  between  visualization  needed  for  situation  awareness  versus  option  awareness  is 
still  emerging.  Significantly  less  effort  has  centered  on  visualizing  the  robustness  of  different 
options  and  interacting  with  the  option  awareness  data  in  a  meaningful  and  comprehensible  way. 
The  research  literature  in  this  area  is  very  recent,  and  scanty. 

We  first  discuss  design  principles  proposed  for  effective  option  awareness  and  some  new 
interaction  and  visualization  techniques.  We  then  present  research  on  bringing  option  awareness 
visualizations  to  a  broader  audience  (to  users  with  strong  domain  knowledge  who  may  not  be 
statisticians). 

4.1.1.  Design  principles  for  effective  option  awareness  visualizations 

Researchers  have  developed  many  design  principles  to  undergird  situation  awareness  visualizations 
(Endsley,  Bolte,  &  Jones,  2003).  However,  Drury,  Pfaff,  Klein,  and  Liu  first  proposed  a  set  of 
visualization  design  principles  to  support  option  awareness  (Drury,  Pfaff,  Klein,  &  Liu,  2013).  These 
principles  emerged  from  multiple  experiments.  Figure  9  gives  an  example  of  an  interactive  box  plot 
visualization  used  in  some  of  these  experiments. 
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User  interface  example  showing  (A)  frequency  distributions  of  outcomes  for  top  six  options,  (E)  sorting  by  rank  or 
quantity  of  resources,  and  (C)  controls  to  customize  the  weighting  strategy  used  to  rank  options,  including  four  presets. 


Figure  9.  Option  awareness  visualization  used  in  experiments  informing  design  principles.  Adapted 
with  permission  from  Drury,  Pfaff,  Klein,  &  Liu  (2013). 


Drury  et  al.'s  design  principles  are  both  interaction  oriented  (i.e.,  allow  users  to  specify  the  values 
for  the  input  parameters,  to  add  their  own  weights  to  each  criterion,  and  to  filter  data  and  sort 
options)  and  visualization  oriented  (i.e.,  show  only  the  top  few  options  and  show  the  outcome 
under  a  variety  of  conditions  rather  than  a  single  set  of  most  likely  conditions).  Their  contributions 
and  insights  cover  much  of  option  awareness  level  1  and  just  begin  to  address  option  awareness 
level  2.  These  represent  the  first  visualization  design  principles  specifically  designed  for  option 
awareness,  and  will  surely  evolve  as  option  awareness  concepts  become  more  widely  adopted  and 
researchers  learn  more  from  real-world  use  of  their  models  and  visualizations. 

4.1.2.  New  interaction  and  visualization  techniques 

As  described  in  section  3,  many  new  visualization  capabilities  have  begun  to  emerge  in  the  field  of 
visual  analytics.  These  capabilities  align  nicely  with  some  of  the  underlying  needs  in  the  area  of 
option  awareness,  as  visualization  of  uncertainty  is  an  essential  part  of  option  awareness.  These 
interactive  visualizations  provide  new  mechanisms  for  achieving  option  awareness.  However,  much 
remains  to  be  done  to  effectively  visualize  option  awareness  data. 
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In  the  remainder  of  this  section  we  describe  some  visualization  techniques  that  reflect  the  latest 
thinking  about  visualizing  uncertainty:  a  capability  essential  to  achieving  option  awareness.  Heinzl 
(2012)  gives  a  sampling  of  the  latest  visualization  and  interaction  techniques,  some  of  which  could 
be  used  to  obtain  option  awareness  level  1  because  they  relate  to  visualizing  the  distribution  of 
values.  Others  provide  an  analysis  of  the  parameter  space  -  a  type  of  visualization  that  closely 
aligns  with  option  awareness  level  2  and  with  understanding  the  factors  underlying  the  robustness 
of  each  option. 

Most  of  the  latest  techniques  related  to  option  awareness  level  1  extend  or  combine  previously 
known  techniques.  Pang's  (2012)  perspective  on  the  field  of  uncertainty  visualization  covered 
visualization  techniques  such  as  histograms,  box  plots,  uncertainty  glyphs,  embellishments, 
pseudo-coloring,  transparency,  and  fuzziness  and  dust  clouds.  Potter  (2012)  described  visualization 
techniques  that  directly  relate  to  option  awareness  level  1  and  visualization  of  the  distribution  of 
values,  such  as  error  bars,  spreadsheets,  and  many  variations  on  box  plots  (including  additions  for 
aesthetics,  value  prevalence,  sample  size,  confidence  levels,  moments,  and  modality).  The  newest 
visualization  technique  she  discussed  was  the  summary  plot  (Figure  10)  which  combines  four 
related  plots  into  a  single  plot  that  presents  more  information  than  any  of  the  visualizations  would 
alone.  Potter,  Kniss,  Riesenfeld,  and  Johnson  (2010)  describe  the  summary  plot  as  an  advanced 
visualization  technique  that  experts  could  use  to  more  fully  understand  the  robustness  of  each 
option.  Such  plots  yield  even  more  information  on  the  comparative  robustness  of  different  options 
to  viewers  with  a  strong  statistics  background.  However,  as  Potter  points  out,  users  need  new 
visual  metaphors  to  help  visualize  uncertainty. 
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Figure  10.  Summary  plot.  Adapted  with  permission  from  Potter,  Kniss,  Riesenfeld,  &  Johnson 

(2010). 
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The  latest  visualization  techniques  related  to  option  awareness  level  2  tend  to  center  on  the 
interactions  among  multiple  linked  views  of  the  data.  A  good  example  of  this  additional  interaction 
is  Heinzl's  (2012)  visualization,  which  assists  car  designers  in  such  tasks  as  selecting  the  fuel 
injection  timing  and  engine  speed  in  order  to  meet  their  targets  with  respect  to  torque  and  fuel 
consumption.  The  visualizations  used  include  scatterplots  and  parallel  coordinate  plots  with  box 
plots  overlaid,  and  incorporate  multiple  linked  views.  Figure  11  shows  examples  of  these 
visualizations.  Note  that  the  visualizations  in  this  example  show  multiple  aspects  of  the  option  to 
be  selected  and  two  criteria  for  evaluating  the  goodness. 


(C) 


Figure  11.  Scatter  plots  and  parallel  coordinate  plots  used  for  car  design.  Adapted  with  permission 
from  Berger,  Piringer,  Filzmoser,  &  Groller  (2011). 


Some  researchers  have  begun  to  address  ways  of  promoting  better  understanding  of  the  space  of 
possible  futures  as  a  scenario  plays  out  over  time.  One  tool,  World  Lines,  does  this  by  using  a 
horizontal  tree  visualization  below  a  timeline  (Waser,  Fuchs,  Ribicic,  Schindler,  Bloschl,  &  Groller, 
2010).  As  the  user  selects  a  new  action  to  try  at  a  particular  point  in  time,  a  new  branch  grows  out 
of  that  tree.  The  user  can  see  multiple  scenarios  (branches  in  the  tree)  in  a  single  view  and 
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compare  how  they  develop  over  time.  The  authors  use  the  example  of  a  flooding  scenario  in  which 
users  must  decide  how  to  place  sandbags  in  a  way  that  minimizes  the  number  of  flooded  buildings 
and  uses  as  few  sandbags  as  possible.  World  Lines  does  not  necessarily  provide  option  awareness 
level  1  or  2,  since  it  does  not  show  the  robustness  or  factors  underlying  the  goodness  of  the  chosen 
options.  However,  it  does  allow  users  to  steer  the  modeling  by  specifying  input  parameters  at 
particular  points  in  time  and  to  try  new  scenarios.  This  may  begin  to  provide  support  for  option 
awareness  level  3.  The  tool  also  represents  a  step  toward  an  ability  to  provide  more  information  on 
the  temporal  dimension  of  option  awareness  data.  Future  research  should  explore  this  area 
further. 

Most  of  the  tools  described  in  the  research  literature  are  best  suited  for  experts  with  statistical 
knowledge.  Engineers  can  use  these  interactive  visualization  tools  and  approaches,  but  the  domain 
expert  would  require  training  on  a  number  of  different  statistical  concepts  to  understand  the 
resulting  visualizations.  In  the  next  section  we  examine  how  innovative  tools  might  reduce  the 
necessary  training  and  provide  more  intuitive  visualizations  that  would  help  make  option 
awareness  more  accessible  to  domain  experts  who  do  not  have  a  background  in  statistics. 

4.1.3.  Bringing  capabilities  to  a  wider  audience 

Visualization  could  greatly  assist  decision  makers  who  could  benefit  from  understanding  the 
robustness  of  their  options  but  aren't  statisticians.  The  visualization  community  has  recently 
recognized  the  importance  of  filling  this  gap  because  more  and  more  mainstream  audiences  now 
use  interactive  visualizations.  In  fact,  a  recent  master's  thesis  (Danziger,  2008)  described  creating 
"Information  Visualization  for  the  People"  to  address  these  opportunities. 

Media  organizations  provide  a  source  of  inspiration.  News  media  must  rapidly  build  visualizations 
that  a  diverse  population  can  easily  interact  with  and  understand.  For  example,  the  New  York 
Times  and  Washington  Post  now  have  interactive  visualization  departments.  The  Washington  Post 
recently  published  one  such  visualization  (Andrews,  Cameron,  &  Keating,  2013).  This  interactive 
visualization  could  inspire  future  option  awareness  level  1  visualizations  because  the  data  has  the 
characteristics  needed:  a  distribution  of  values  (for  a  given  medical  procedure).  The  equivalent 
data  in  our  example  relates  to  the  distribution  of  outcomes  for  a  given  option.  In  the  Washington 
Post  visualization,  they  chose  to  show  the  lowest  value,  highest  value,  and  average  value.  By 
selecting  a  particular  state  from  the  drop-down  menu  at  the  top,  users  can  see  where  a  particular 
state  appears  in  the  distribution,  revealing  how  that  state  compares  to  the  others  in  terms  of  being 
closer  to  the  minimum  or  maximum  or  closer  to  the  average. 

A  second  example,  this  one  from  the  New  York  Times,  provides  a  visualization  that  could  serve  as 
an  inspiration  for  providing  option  awareness  level  2.  The  New  York  Times  created  this  visualization 
(Bostock  &  Carter,  2012)  in  the  Fall  of  2012  during  the  U.S.  presidential  race.  The  example  shows 
the  rough  equivalent  of  an  interactive  decision  tree.  A  user  can  drill  down  into  the  data  by  mousing 
over  paths  in  the  tree  to  obtain  more  detail,  and  can  filter  the  data  by  selecting  a  particular  path  or 
by  selecting  the  outcome  in  a  particular  state  shown  at  the  top.  The  filtering  prunes  the  tree  to 
show  the  remaining  paths  to  victory  for  each  presidential  candidate.  Similar  visualizations  and 
interactions  could  provide  level  2  option  awareness  to  users  if  populated  with  different  data. 
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These  visualizations,  particularly  in  their  interactive  form,  increase  the  understandability  of 
complex  data,  thus  reducing  the  human  cognitive  bottleneck.  The  data  may  be  complicated,  but 
the  visualizations  need  not  be.  Researchers  must  apply  this  approach  of  simplifying  complex  data 
as  much  as  possible,  but  no  more,  to  option  awareness  so  that  the  visualizations  can  be  used  by  a 
wider  audience.  News  media  organizations  such  as  those  mentioned  here  probably  have  especially 
valuable  information  regarding  how  much  to  simplify  an  analysis  to  make  the  process  clear  to  the 
average  person  who  does  not  have  a  strong  statistics  background. 

However,  research  must  still  determine  the  effectiveness  of  such  visualization  techniques  in  the 
context  of  supporting  robust  decision  making.  To  generate  option  awareness,  decision  support 
scenarios  may  call  for  more  complex  information  than  that  used  in  the  examples  above. 
Nonetheless,  these  visualizations  provide  important  examples  of  techniques  that  do  not  require 
explanation  and  instead  enable  users  to  focus  only  on  understanding  the  data.  The  key  is  to  find  an 
appropriate  balance. 

4.2.  Gaps 

All  of  the  option  awareness  concepts,  new  visualization  design  ideas,  the  ability  to  simplify  those 
designs,  and  the  underlying  design  principles  are  evolving  and  would  benefit  from  greater  insight. 
New,  more  effective  visualization  techniques  are  needed  for  all  levels  of  option  awareness, 
particularly  for  level  3.  The  design  principles  must  evolve  together  with  the  concepts  and  practice, 
based  on  use  in  the  real  world.  Research  should  also  seek  to  develop  methods  for  bringing  all  three 
levels  of  option  awareness  together  into  a  single  integrated  tool:  one  that  allows  users  to  progress 
easily  from  one  level  of  option  awareness  to  the  next.  As  the  basics  evolve,  these  visualizations 
must  eventually  be  adapted  to  extend  beyond  the  desktop  to  visualizations  appropriate  for 
decision  makers  who  use  tablets  or  mobile  devices  to  display  data;  in  other  words,  to  move  the 
capabilities  out  of  command  center-like  environments  and  into  the  hands  of  users  in  the  field. 


5.  Call  to  Action 

To  bring  the  new  technologies  and  methods  into  operational  use  all  of  these  techniques  must 
coalesce  and  the  emerging  set  of  design  principles  must  evolve.  Widespread  use  of  visualization 
tools  depends  on  advanced  design  principles  based  on  observations  made  in  the  field  rather  than 
only  in  a  laboratory. 

In  addition  to  systems  based  on  these  advanced  visualization  design  principles,  users  will  need  to 
understand  the  underlying  analysis  that  generated  the  data  shown  in  the  interactive  visualizations, 
and  will  need  to  be  trained  on  how  to  apply  these  visualizations  for  decision  making.  User  training 
and  experience,  combined  with  better  design  principles  and  tools,  could  avoid  many  potential 
pitfalls  in  the  use  of  visualizations.  Further,  as  visualizations  evolve,  researchers  must  determine 
how  much  to  simplify  the  complexity  of  the  underlying  analysis  and  visualization  techniques. 
Effective  visualizations  must  achieve  a  balance  between  supplying  enough  detail  to  support  good 
decisions  while  not  displaying  so  much  detail  that  users  dismiss  the  tools  as  too  complex. 

In  the  meantime,  developers  should  seek  to  smooth  the  transitions  from  situation  awareness 
displays  into  option  awareness  displays.  Tools  are  also  needed  that  allow  users  to  progress  from 
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one  level  of  option  awareness  to  the  next.  As  a  situation  plays  out  some  of  the  uncertainty  will 
disappear,  which  calls  for  rapid  feedback  into  the  tools  to  update  underlying  assumptions  and 
uncertainties.  Option  awareness  visualizations  must  also  update  option  comparisons  as  uncertainty 
decreases  to  help  users  know  when  to  adapt  their  mitigating  actions  as  a  situation  evolves. 

The  future  of  option  awareness  visualization  is  wide  open,  with  research  only  starting  to  focus  on 
option  awareness  as  a  distinct  concept.  Realizing  these  capabilities  in  future  visual  analytics  tools 
offers  great  potential  benefits  to  decision  makers  faced  with  uncertainty. 
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16  Enabling  operational  users  to  fully  exploit  the  tools, 
data,  and  analytics  to  shape  potential  futures1 

Alicio  Ruvinsky,  Rachel  Hingst,  Mark  Hoffman  &  Brian  Kettler 
Lockheed  Martin  Advanced  Technology  Labs 


1.  Overview 

Human,  social,  cultural,  and  behavioral  (HSCB)  modeling  with  forecasting  and  mitigation-option 
capabilities  improves  operators'  ability  to  understand  and  shape  the  operational  environment. 
Sociocultural  data  and  analysis  assist  in  the  identification  of  crises  further  "left  of  bang,"  allowing 
consideration  of  more  mitigation  options  and  more  time  to  assess  possible  ramifications  (Flynn, 
Sisco,  &  Ellis,  2012).  As  a  greater  number  of  HSCB-driven  mitigation  technologies  improve  and  are 
deployed,  more  operational  users  will  require  training  on  these  tools,  data,  and  analytics  to 
mitigate  and  shape  potential  futures. 

Limited  time,  tools,  and  capacity  currently  constrain  the  number  of  prospective  courses  of  action 
(COAs)  that  operational  planners  can  generate  and  assess  as  they  try  to  mitigate  or  avert  potential 
crises.  Processes  implemented  today  are  highly  manually  intensive  and  seldom  integrated 
(computationally)  with  HSCB  knowledge.  As  HSCB-driven  mitigation  technologies  continue  to 
evolve  at  a  rapid  pace,  the  quality  and  kind  of  information  made  available  to  users  and  the 
potential  utility  of  this  content  will  begin  to  blur  the  traditional  "lanes"  between  current  career 
fields  such  as  intelligence,  operations  research,  and  operational  planning.  This,  in  turn,  will  require 
innovations  in  training  to  prepare  users  to  effectively  recognize  and  leverage  content  relevant  to 
analyzing  COAs  for  mitigation.  For  example,  training  on  model  interpretation,  typically 
concentrated  in  particular  career  fields  such  as  operations  research,  must  be  expanded  to  ensure 
that  more  operational  users  can  effectively  apply  emerging  mitigation  options. 

As  the  proliferation  of  HSCB  knowledge  makes  COA  mitigation  capabilities  more  accessible, 
innovative  training  for  the  skills  and  knowledge  needed  to  perform  COA  mitigation  tasks  becomes 
imperative.  This  chapter  focuses  primarily  on  the  mitigation  tasks  related  to  COA  development, 
COA  analysis,  and  COA  assessment.  For  each  key  mitigation  task,  trainers  must  consider  user  skill 
(ability  necessary  to  perform  the  task)  and  knowledge  (facts,  concepts,  and  principles  required  to 
perform  the  task),  as  defined  in  the  Military  Handbook  29612-2A  (MIL-HDBK-29612-2A,  2001).  An 
initial  analysis  of  the  functional  tasks  involved  in  mitigation  highlights  several  considerations,  such 
as  key  training  objectives  and  ideal  training  formats  (e.g.,  computer  based  training,  classroom,  or 
exercise). 


1  Approved  for  Public  Release;  Distribution  Unlimited.  Case  Number  14-2487 
This  work  was  supported  by  Department  of  Defense  Contract  W15P7T-13-C-F600 
Copyright  ©  2014  The  MITRE  Corporation. 


Tools ,  data,  and  analytics  to  shape  potential  futures  |  431 


This  chapter  begins  with  an  overview  of  the  HSCB  knowledge  and  skills  necessary  to  perform 
mitigation.  We  present  a  use  case  detailing  the  processes  involved  in  development,  analysis,  and 
assessment  of  COAs.  From  the  context  generated  by  the  use  case,  we  describe  training  for 
mitigation  by  addressing  training  for  each  of  those  processes.  We  present  each  of  the  processes  in 
terms  of  the  knowledge  and  skills  needed,  the  implications  for  training,  and  the  state  of  the 
practice. 

1.1.  Knowledge  and  Skills  for  Mitigation 

In  general,  HSCB  knowledge  and  the  skills  for  acquiring,  understanding,  applying  and  maintaining  it 
can  facilitate  each  mitigation  task.  HSCB  knowledge  spans  general  sociocultural  information,  such 
as  current  and  historical  characteristics  of  a  geographic  region  (country,  city,  village,  etc.),  groups, 
and  even  individuals.  This  information  builds  context  for  and  supports  computational  social  science 
and  other  analytical  capabilities  designed  to  anticipate  the  effects  (including  nth-order  effects)  of 
particular  COAs  and  assess  their  impact.  It  also  provides  greater  context  to  enable  mitigation  of  a 
forecasted  event  (e.g.,  understand  the  rationale  behind  the  forecast  and  the  significance  and 
potential  impact  of  the  event  to  mitigate).  Aspects  of  HSCB  knowledge  include  the  PMESII  (political, 
military,  economic,  social,  informational,  infrastructure)  conditions  in  a  region,  as  well  as  the  goals, 
standards,  preferences,  and  beliefs  of  groups  and  key  influential  individuals  (e.g.,  leaders)  in  the 
region,  and  other  key  regional  players  and  their  objectives.  Training  for  mitigation  cultivates  the 
special  knowledge  and  skills  that  planners  need  to  thoroughly  understand  applicable  U.S. 
objectives  and  priorities  when  considering  diplomatic,  information,  military,  and  economic  (DIME) 
options  for  COAs,  resources  available  to  execute  them,  the  effects  of  the  COAs,  and  measures  of 
effectiveness  and  performance. 

Operators  need  many  skills  for  mitigation  and  must  be  trained  to  acquire  them,  but  we  focus  here 
on  those  skills  most  influenced  by  recent  improvements  in  HSCB  capabilities.  All  planners  and 
analysts  considering  mitigation  options  must  learn  to  rapidly  ingest  available  information  about  the 
operational  environment.  To  properly  harness  emerging  HSCB  capabilities,  they  must  know  how  to 
use  the  tools,  understand  the  quantitative  and  qualitative  aspects  of  social  science  modeling,  and 
recognize  the  benefits  and  limitations  of  the  data  sources  used  by  HSCB  tools. 

To  more  clearly  identify  requirements  for  mitigation  training,  we  consider  the  mitigation  process 
from  two  perspectives:  (1)  the  kinds  of  HSCB  knowledge  needed  to  perform  mitigation  analysis, 
and  (2)  the  skills  needed  to  incorporate  the  HSCB  dimensions  of  a  geospatial  region  into  the 
planning  process.  We  categorize  these  in  terms  of  the  development,  analysis,  and  assessment 
phases. 

The  next  section  of  this  chapter  presents  a  mitigation  use  case  that  offers  a  notional  background 
for  describing  the  knowledge  and  skills  needed  for  each  of  the  phases  of  mitigation.  It  discusses  the 
three  phases  of  constructing  a  Theater  Campaign  Plan  (TCP),  and  highlights  some  emerging  HSCB 
technologies  beginning  to  have  an  impact  on  this  process.  Subsequent  sections  summarize  the 
HSCB  knowledge  and  skills  required  for  each  phase  of  mitigation.  Finally,  we  discuss  the  objectives 
for  training  to  support  planners  and  analysts  in  developing  the  capabilities  required  to  perform 
mitigation  effectively. 
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2.  Mitigation  Use  Case:  Theater  Campaign  Plan 

As  an  example  of  the  skills  and  knowledge  required  from  operators  to  perform  mitigation,  we 
describe  the  TCP:  a  strategic  plan  designed  to  mitigate  instability  and  to  achieve  other  desired  end 
states  in  a  Combatant  Command's  (CCMD's)  area  of  responsibility  (AOR).  An  AOR  includes  a 
number  of  countries  that  may  exhibit  considerable  diversity.  For  example,  U.S.  Pacific  Command 
(PACOM)  spans  36  countries  around  the  Pacific  Rim,  including  Australia,  China,  and  Fiji.  Planning  in 
the  PACOM  AOR  is  therefore  a  challenging  problem  that  calls  for  considerable  FISCB  knowledge, 
skills,  and  tools  (see  Figure  1).  U.S.  goals  in  a  geographic  AOR  include  the  prevention  and  mitigation 
of  regional  instability.  For  peacetime  operations,  the  TCP  and  country  plans  represent  longer  range, 
multi-year  allocations  of  resources  to  appropriate  DIME  actions. 


Risk  Assessment/Trade-offs; 
Capability  Gaps 


National  Guidance 


Commander's  Guidance 


Execution 


Theater  Campaign  Plan 


Figure  1.  National  Defense  University  model  of  the  strategic  planning  process. 


The  TCP  process  not  only  reveals  opportunities  and  requirements  for  additional  automation2,  but 
also  helps  illustrate  additional  requirements  for  human  competencies  (knowledge  and  skills)  that 
must  be  conveyed  through  training.  The  following  description  of  the  TCP  is  based  on  official 
planning  guidance  (Theater  campaign  planning,  2012)  and  on  relevant  discussions  with  CCMDs  that 


2  DARPA  Integrated  Crisis  Early  Warning  System  (ICEWS)  effort  (O'Brien,  2010;  Kettler  &  Floffman,  2012)  was 
initially  targeted  to  aid  CCMDs  to  develop  and  assess  the  Theater  Security  Cooperation  Plan  (TSCP),  a  key  part 
of  the  TCP  that  focuses  on  bilateral  DIME  actions  such  as  joint  training  exercises  between  the  U.S.  and  another 
country. 
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took  place  as  part  of  the  Defense  Advanced  Research  Projects  Agency  (DARPA)  Integrated  Crisis 
Early  Warning  System  (ICEWS)  program. 

Inputs  to  the  TCP  include  global  U.S.  priorities  as  articulated  in  the  Guidance  for  the  Employment  of 
the  Force  (GEF)  and  Joint  Strategic  Capabilities  Plan  (JSCP).  The  GEF  specifies  broad  strategic 
theater  end  states,  looking  out  5  to  10  years.  TCPs  have  a  strategic  focus  on  the  longer  term,  rather 
than  on  a  specific  operation  (e.g.,  a  combat  operation,  relief  for  a  particular  disaster,  etc.).  They 
also  emphasize  planning  for  the  "steady  state"  by  describing  DIME  actions  and  resources  required 
to  make  progress  on  longer  term  objectives,  including  keeping  the  peace  by  mitigating  potential 
instability.  TCPs  are  not  static  and  must  be  updated  if  the  situation  in  the  AOR  changes,  for  instance 
if  resources  become  unavailable  or  a  natural  disaster  occurs.  For  example,  the  2005  tsunami  in 
Indonesia  required  considerable  attention  and  resources  from  PACOM,  given  the  scale  of 
devastation  and  the  longer  term  potential  for  adverse  impact  on  regional  stability.  TCPs  require 
coordination  across  the  DoD  (e.g.,  individual  services  supporting  the  CCMDs),  non-DoD  U.S. 
Government  agencies  (e.g.,  the  State  Department,  U.S.  Agency  for  International  Development),  the 
individual  countries  in  the  AOR,  allies,  and  nongovernmental  organizations  (NGOs).  Many  of  those 
entities  are  involved  in  the  actual  execution  of  the  TCP  -  for  example,  providing  resources  -  with 
the  CCMDs  responsible  for  overall  planning,  execution,  and  assessment  of  the  TCP. 

An  early  task  in  TCP  planning  is  COA  development,  which  focuses  on  mission  analysis  and  on 
assimilating  high-level  guidance  from  documents  such  as  the  GEF  and  JSCP,  and  includes  identifying 
desired  theater  strategic  end  states  projected  5  to  10  years  in  the  future.  The  planners  integrate 
guidance  and  supporting  information  into  (a)  intermediate  military  objectives  (IMOs)3  leading 
toward  end  states,  (b)  key  planning  assumptions,  and  (c)  available  resources.  Intelligence  support 
to  TCP  planners  identify  and  describe  threats  and  opportunities  associated  with  desired  end  states, 
including  regional  PMESII  factors,  the  potential  COAs  of  countries  outside  the  AOR  and  hostile 
countries  inside  the  AOR,  and  transnational  actors  that  may  have  their  own  interests  and  objectives 
in  the  AOR. 

ICEWS  forecasts  of  country  instability,  which  provide  early  warning  of  potential  hot  spots  where 
the  TCP  planners  may  need  to  consider  COAs  for  mitigation  efforts,  represent  one  example  of 
recent  HSCB-enabled  technology  that  can  have  an  impact  in  the  area  of  mitigation.  Recent  work  on 
supplementing  ICEWS  forecasts  with  details  automatically  mined  from  ongoing  monitoring  of  open 
media  (newsfeeds,  social  media,  etc.)  and  classified  sources  may  provide  additional  guidance  for 
refining  TCP  development  by  elaborating  and  verifying  the  actionable  details  of  a  potential 
instability  event  of  interest  (EOI).  For  example,  understanding  which  groups  in  a  country  might  be 
fomenting  unrest  and  why  they  are  doing  so  could  help  avoid  an  event  by  suggesting  more  specific 
and  effective  COAs  that  the  TCP  may  identify  with  respect  to  a  peacekeeping  IMO. 


3  Examples  of  IMOs  include  building  partner  (country)  capacities,  gaining  access,  maintaining  relationships, 
conducting  security  force  assistance,  supporting  ongoing  operations,  etc.  Like  all  military  objectives,  IMOs  are 
ideally  outcome-  vs.  process-oriented,  time-bound,  and  measurable. 
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Mindful  of  the  threats  and  opportunities  in  the  AOR,  the  CCMD  next  develops  a  TCP  around  IMOs 
that  contribute  to  the  desired  end  states.  Initially  the  ICEWS  forecasting  capabilities  (iCAST)  were 
intended  to  help  CCMDs  determine  where  to  focus  their  TCP  portfolios,  given  their  objectives, 
repertoire  of  potential  DIME  actions,4  and  available  resources. 

COA  development  centers  on  assimilating  guidance  into  the  identification  of  a  desired  end  state 
and  a  strategic  plan  for  realizing  that  end  state.  Training  for  COA  development  must  advance 
students'  knowledge  and  skills  in  identifying  a  desired  end  state  consistent  with  guidance,  relevant 
objectives,  and  possible  actions  (i.e.,  potential  COAs)  to  accomplish  the  objectives  based  on 
awareness  of  the  current  state  and  the  expected  impact  of  possible  COAs.  While  most  planning 
courses  currently  teach  these  fundamentals  to  some  degree,  they  must  be  extended  to  incorporate 
emerging  HSCB  capabilities.  For  example,  TCP  planners  should  learn  to  better  incorporate  social 
science  knowledge  about  the  populations  affected  to  increase  the  number  of  potential  COAs  that 
might  achieve  key  objectives. 

Determining  the  best  COAs  corresponding  to  TCP  objectives  requires  analysis  of  potential  COAs  to 
project  and  assess  the  likely  impact.  Ideally,  this  includes  modeling  the  actions  to  provide  a  deeper 
understanding  of  potential  impact.  Such  modeling  can  be  difficult,  since  the  actions  themselves 
may  be  ill  specified.  Their  effects  may  be  conditional  (i.e.,  differ  depending  on  the  situation), 
extended  in  time,  subject  to  interaction  with  the  effects  of  other  actions  that  overlap  in  time,  or 
dependent  on  unknown  factors.  Some  contemporaneous  actions  may  be  unknown  (e.g.,  the  DIME 
actions  by  other  world  powers  with  potentially  competing  agendas).  Actions  may  have  nth-order 
effects  that  are  neither  well  known  nor  easy  to  mine  from  incomplete  or  noisy  historical  data. 

To  analyze  COAs  for  DIME  objectives,  planners  must  understand  the  impact  an  action  may  have  on 
the  operational  environment  (see  Figure  2).  The  ICEWS  project  built  Bayesian  models  based  on  the 
input  of  subject  matter  experts  (SMEs),  such  as  social  scientists,  country  analysts,  etc.,  to  explore 
the  impact  of  DIME  actions.  The  key  SME  tasks  included  defining  DIME  actions  (e.g.,  what 
constitutes  an  economic  action  such  as  "Provide  Financial  Aid"),  eliciting  contextual  variables  that 
determine  the  conditional  effects  of  a  given  action  (e.g.,  Action  X  can  have  one  set  of  effects  in 
Country  X  and  different  effects  in  Country  V),  and  "linking"  variables  that  capture  the  effects  of 
DIME  actions  (e.g.,  the  change  in  rate  of  violent  events  in  a  country).  For  example,  SMEs  associated 
an  economic  action  such  as  "Provide  Financial  Aid"  with  context  variables  such  as  the  capacity  of 
government,  influence  of  trading  partners,  distribution/delivery  mechanisms,  etc.  The  SMEs 
correlated  these  context  variables  with  effect  variables,  such  as  impact  on  Cooperation  (e.g.,  level 
of  cooperative  events)  and  Discord  (e.g.,  level  of  violent  events). 


4  Possible  DIME  actions  include  activities  such  as  visits  by  high-level  personnel,  bilateral  conferences,  bilateral 
training  exercises,  port  visits  (for  diplomatic,  show-of-force,  and  economic  impact),  medical  assistance, 
engineering  project  assistance,  etc. 
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Figure  2.  SMEs  defined  (a)  the  DIME  action  ("Action  1"),  (b)  relevant  contextual  variables  ("Action 
Context"),  and  (c)  effects  of  DIME  actions  ("Action  Effects). 


COA  analysis  assumes  that  the  development  phase  has  generated  one  or  more  potential  COAs;  the 
analysis  then  focuses  on  projecting  and  assessing  the  future  impact  of  the  actions  defined  in  those 
COAs.  Training  for  COA  analysis  must  emphasize  acquisition  of  knowledge  and  skills  for  identifying 
context  variables  relevant  to  the  action  and  effect  variables  influenced  by  the  action, 
understanding  the  relationship  between  the  context  and  effect  variables,  and  increasing  awareness 
of  sensitivities  such  as  conditional  or  interaction  effects.  Advances  in  HSCB  technologies,  including 
"what-if"  forecasting  and  variable  identification,  have  built  on  some  of  the  lessons  learned  from 
the  intensive  modeling  exercises  with  SMEs,  making  the  information  more  operationally  accessible. 
To  take  advantage  of  these  technologies,  TCP  planners  need  to  be  trained  on  HSCB  models  and 
other  relevant  tools. 

COA  assessment  for  TCPs  often  occurs  in  parallel  with  COA  development,  as  different  parts  of  the 
TCP  may  be  in  different  stages  of  planning  and  ongoing  execution.  Assessment  may  be  triggered  by 
regular  review  timetables,  commanders'  questions  or  directives,  developing  world  events, 
emergent  threats  or  opportunities,  the  completion  (or  failure)  of  DIME  actions,  execution  of  the 
TCP,  etc.  Assessment  includes  applying  various  measures  to  evaluate  the  performance  of  planned 
tasks/actions  (i.e.,  did  the  action  take  place  as  planned?).  For  example,  did  a  port  visit  occur  or  was 
it  prevented  by  weather  or  other  factors?  Was  the  action  only  partially  completed?  If  an  action 
executed  as  planned,  did  it  have  the  desired  diplomatic,  military,  political,  or  economic  impact, 
contributing  to  a  higher  level  desired  end  state?  For  example,  perhaps  the  port  visit  was  designed 
to  improve  public  support  for  the  United  States,  but  actually  spawned  anti-U.S.  protests.  Finally, 
the  assessment  must  cover  the  cost  of  performance  versus  the  effectiveness  of  the  action:  was  the 
resource  investment  worthwhile  compared  to  alternative  actions? 
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Assessments  require  significant  resources  to  capture  the  relevant  information,  analyze  it, 
assimilate  it,  and  archive  it  for  future  use.  Assessments  demand  manpower,  tools,  and  data,  and 
should  be  performed  in  a  timely  fashion  while  information  is  fresh.  Tools  have  typically  lagged 
behind  the  need  and  have  been  limited  to  databases,  spreadsheets,  and  other  ad  hoc  methods  that 
impeded  information  integration,  sharing,  and  reuse.  Typically  it  is  easier  to  collect  information 
pertaining  to  measuring  performance  (what  was  done)  vs.  measuring  effectiveness  (what  was 
achieved  in  terms  of  actual  vs.  desired  effects). 

Operational  deployment  of  capabilities  such  as  those  offered  by  ICEWS  allows  the  user  to  leverage 
open  media  (e.g.,  news  reports,  social  media,5  and  other  sources  such  as  financial  market 
indicators)  to  augment  information  gathered  by  CCMD  assessment  staff  (Sanchack,  2012).  For 
many  countries,  these  sources  provide  a  wealth  of  indicators  relative  to  the  issues  that  concern  the 
population  of  a  country  or  city  and  their  sentiment  about  those  issues.  News  reports  can  provide 
useful  indications  regarding  the  effects  of  a  DIME  action.  For  example,  while  a  port  visit  is  ongoing, 
analysts  can  inexpensively  detect  protests  or  other  reactions  among  the  populace  by  examining 
news  reports  from  multiple  sources.  This  helps  define  metrics  for  assessing  effectiveness  of  the 
current  TCP  and  for  building  computational  models  of  DIME  actions  in  the  future. 

Training  for  COA  assessment  should  enable  students  to  gain  knowledge  and  skills  for  evaluating 
COA  execution;  measuring  COA  impact  based  on  indicators  in  HSCB  knowledge,  including  analysis 
of  open  media;  and  assessing  COA  suitability  in  terms  of  resource  availability  and  consumption, 
cost,  and  other  measures. 

3.  Knowledge  and  Skills  Requirements  for  Functional  Mitigation  Tasks 

This  section  presents  an  initial  analysis  of  the  knowledge  and  skills  needed  to  perform  the 
functional  mitigation  tasks  of  COA  development,  COA  analysis,  and  COA  assessment.  The  discussion 
will  guide  training  considerations  by  identifying  the  learning  objectives  that  foster  the  requisite 
human  competencies.  In  this  section,  we  describe  the  knowledge  that  informs  all  areas  of 
mitigation  tasking  and  then  decompose  each  mitigation  task  to  identify  the  particular  skills  needed 
to  perform  them. 

3.1.  Knowledge  for  COA  Development,  Analysis,  and  Assessment 

COA  development,  analysis,  and  assessment  require  similar  kinds  of  knowledge,  although  each 
uses  that  knowledge  in  a  unique  manner.  To  perform  mitigation  tasks  analysts  need  (a)  social 
science  knowledge  resulting  from  integration  of  information  sources  such  as  population-centric 
information  and  sociocultural  information,  and  (b)  knowledge  of  modeling  (including  underlying 
theories  and  data)  and  model  results. 

Social  science  knowledge  generated  from  population-centric  information  and  sociocultural 
information  describes  the  HSCB  dimensions  of  the  operational  environment  that  are  essential  for 


5  Via  the  newer  iSENT  social  media  (Twitter,  blog,  etc.)  ingest  and  analytic  capabilities,  which  include 
sentiment  and  influence  analysis. 
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mitigation  tasks.  Population-centric  information  details  the  demographics  of  the  population,  while 
sociocultural  information  characterizes  the  relationships  among  contextual  factors  and  individuals 
or  populations.  This  includes  such  information  as  where  people  are  located  in  a  geographical  space, 
group  associations,  political  or  religious  affiliations,  economic  conditions,  social  and  cultural 
expectations  regarding  how  people  interact  with  their  environment  and  each  other,  and  how 
deviations  from  social  norms  (e.g.,  improper  discarding  of  holy  items,  inappropriate  interactions 
with  respected  or  authority  individuals)  may  affect  social  responses  such  as  sentiment  or  mood. 
These  data  are  critical  for  developing  high-quality  and  effective  COA  alternatives.  The  alternatives 
built  upon  this  HSCB  perspective  reflect  an  understanding  of  the  commander's  intent  within  the 
sociocultural  context  of  the  local  population,  which  results  in  an  accurate  and  effective  design  for 
realizing  essential  tasks  and,  ultimately,  in  mission  success.  The  integration  of  these  kinds  of 
information  produces  social  science  knowledge  that  enables  useful  understanding  of  the 
correlations  among  contextual  factors  and  their  impact  on  the  operational  environment  to  support 
the  execution  of  necessary  actions  for  mitigation. 

To  effectively  plan  tasks  to  be  performed  within  a  geographic  region,  the  deployed  resources  must 
understand  demographic  characteristics  of  the  local  population.  For  example,  if  the  United  States 
intends  to  provide  food  as  aid  to  a  particular  region,  planners  must  understand  the  relevant  tenets 
of  the  population's  religion  (e.g.,  dietary  or  handling  implications). 

Social  and  cultural  behaviors  can  indicate  group  norms,  where  these  norms  encompass  the  actions 
or  interactions  that  individuals  of  a  particular  social  or  cultural  group  would  expect  from  their 
environment.  These  kinds  of  interactions  can  be  as  simple  as  shaking  hands  when  meeting 
someone  or  as  complicated  as  understanding  an  inherent  social  hierarchy  and  sitting  lower  (i.e., 
closer  to  the  floor)  than  a  perceived  superior.  Understanding  this  type  of  information  facilitates  the 
use  of  other  relevant  data  sources  such  as  social  media  in  order  to  extract  supporting  content  that 
provides  deeper  awareness  and  understanding.  Mitigation  training  needed  to  support  the 
processing  of  HSCB  knowledge  during  COA  development  must  be  based  on  access  to  both 
population-centric  and  sociocultural  information.  For  example,  geolocated  blog  posts  that  reveal 
the  political  perspectives  of  the  people  in  an  area  can  add  depth  to  population-centric  information 
describing  the  political  demographics  of  that  area.  Social  media  offer  a  fertile  platform  for 
acquiring  population-centric  and  sociocultural  information  supporting  various  kinds  of  HSCB 
knowledge.  These  media  can  be  a  valuable  source  of  data  for  sentiment  analysis  -  assessing  the 
attitudes,  emotions,  sentiments,  and  opinions  of  a  population  -  and  mood  shift  analysis,  which 
specifically  considers  the  trends  in  sentiment  or  mood.  Social  media  also  aid  in  trust  assessment  of 
open  source  data.  Trust  is  a  key  element  in  the  effective  functioning  of  groups. 

To  generate  alternative  COAs  in  response  to  a  mission  statement  and  a  strategic  approach, 
planners  need  the  ability  to  understand  what  the  data  convey  about  the  characteristics  and 
configuration  of  a  population,  and  what  the  models  indicate  with  respect  to  the  theoretical 
significance  of  data  trends.  Models  that  depict  how  particular  factors  influence  the  human,  social, 
cultural  and/or  behavioral  dimensions  of  a  population  help  analysts  to  project  an  expected 
consequence  based  on  an  initial  configuration  of  such  factors  and  DIME  actions.  These  types  of 
modeling  tools  provide  the  knowledge  necessary  for  better  understanding  the  relationships  among 
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contextual  factors  within  the  HSCB  space  and  the  impact  those  factors  have  on  the  stability  of  the 
operational  environment.  Exploring  models  and  manipulating  the  factors  they  encapsulate  allow 
analysts  to  (a)  investigate  and  identify  threats  and  opportunities  and  their  effects  on  the  desired 
end  state,  (b)  recognize  high-level  objectives  of  parties  external  to  the  AOR  that  may  contend  or 
otherwise  affect  the  strategic  agenda  of  the  CCMD,  (c)  project  and  evaluate  future  impact  of 
changes  in  contextual  factors,  and  (d)  contribute  to  defining  measures  for  assessment  and 
prioritization  of  COAs. 

HSCB  models  enable  users  to  apply  the  massive  and  complex  data  in  the  HSCB  space  in  order  to 
present  analysts  with  tractable  content  and  enable  them  to  easily  and  quickly  identify  relevant 
information.  However,  all  models  present  a  simplified  view  of  the  actual  world.  To  effectively 
leverage  a  model,  users  must  translate  the  model's  representation  of  the  world  to  fit  the  user's 
perspective.  Understanding  a  model's  output  requires  knowledge  of  the  purpose  and  performance 
of  the  model  and  awareness  of  sociocultural  elements  of  the  environment  such  as  population 
distribution  and  composition,  cultural  norms,  or  religious  affiliations.  Users'  trust  in  the  ability  of  a 
tool  to  help  generate  COAs  will  depend  on  substantiated  evidence  that  the  model  enables  a  valid 
representation  of  the  operational  environment.  Thus,  intelligence  and  operational  planning  experts 
must  be  trained  on  how  HSCB  models  represent  the  operational  environment,  and  how  to  translate 
and  leverage  this  content  for  COA  development.  This  will  aid  users  to  interpret  the  model  results, 
build  trust  in  the  model,  and  gain  awareness  of  factors  in  the  real  world  (based  on  model  analysis) 
that  are  relevant  to  COAs. 

Because  HSCB  models  rely  on  sociocultural  knowledge  and  theories,  users  must  also  learn  that 
some  models  are  more  heuristic  based  (Moss  &  Edmonds,  2005)  than  the  models  currently  applied 
to  operations,  which  typically  provide  detailed  representations  of  potential  kinetic  force-on-force 
outcomes.  Nevertheless,  these  heuristic  models  still  provide  valuable  operational  insight. 

Clear  visualizations  and  model  transparency  are  critical  to  training  in  COA  development. 
Interpreting  visual  representations  of  model  performance,  forecasts,  analysis,  etc.,  must  become 
second  nature  to  the  HSCB  model  user/COA  developer  in  order  to  make  the  results  optimally 
effective  in  assessing  relevant  HSCB  content  and  model  forecasts  quickly.  Training  must  take  into 
account  that  many  potential  users  will  be  qualitative  vs.  quantitative  thinkers;  good  visualization 
tools  must  support  a  range  of  users  with  varied  job  functions  and  modeling  and  social  science 
expertise  (OSD,  2009). 

3.2.  Skills  for  COA  Development 

To  mitigate  or  avert  a  potential  crisis  or  event,  planners  must  first  develop  COAs  that  counter  the 
undesirable  action  and  promote  achievement  of  the  desired  end  state.  Once  planners  have 
analyzed  the  mission  and  generated  high-level  guidance  to  achieve  desired  theater  strategic  end 
states,  the  TCP  identifies  objectives  (i.e.,  IMOs)  that  contribute  to  these  end  states  and  ultimately 
designates  COAs  intended  to  realize  the  TCP  objectives.  COA  developers  such  as  TCP  planners  must 
have  the  knowledge  and  skills  to  identify  contextual  factors  relevant  to  realizing  the  desired  end 
state;  understand  the  potential  COAs  of  friendly,  neutral,  and  adversary  nations  and  transnational 
groups  both  inside  and  outside  the  AOR;  assimilate  guidance  into  the  identification  and  description 
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of  a  desired  end  state;  and  generate  COAs  with  the  expectation  of  realizing  the  desired  end  state. 
This  should  include  the  ability  to  describe  threats  and  opportunities  associated  with  that  end  state, 
identify  relevant  regional  PMESII  factors  in  the  AOR,  define  objectives  (i.e.,  IMOs)  leading  toward 
the  desired  end  state,  recognize  key  planning  assumptions,  and,  finally,  for  each  objective, 
generate  COA(s)  designed  to  realize  the  objective  and  determine  the  available  resources  for 
performing  the  COA(s). 

COA  development  requires  awareness  and  understanding  of  the  desired  or  projected  future.  The 
following  subsections  identify  the  knowledge  and  skills  required  to  perform  each  of  the  four  high- 
level  tasks  comprising  COA  development.  They  then  describe  how  these  knowledge  and  skills 
translate  into  training  implications  for  developing  the  awareness  and  understanding  necessary  to 
perform  these  tasks  effectively. 

3.2.1.  Skills  to  identify  contextual  factors  of  consequence 

Once  the  commander  has  set  a  high-level  strategic  agenda  and  characterized  it  in  terms  of  a 
desired  end  state  and  objectives,  a  COA  developer  must  extrapolate  and  identify  aspects  of  the 
operational  environment  relevant  to  the  agenda.  These  aspects  consist  of  contextual  factors  that 
will  aid  in  assessing  the  current  state,  forecasting  the  future  state,  and  interpreting  the  impact  of 
actions  on  the  future  state. 

While  the  intelligence  career  field  has  built  some  expertise  in  these  skills,  the  planning  community 
(though  supported  by  intelligence)  typically  consists  of  operators  (e.g.,  Air  Force  pilots).  Training  for 
these  individuals  must  cover  emerging  HSCB-enhanced  mitigation  tools  (such  as  social  science 
forecasting  models).  The  ability  to  understand  models  includes  the  ability  to  observe  and  learn  the 
relationships  and  factors  presented  by  a  model  of  an  HSCB  phenomenon.  Planners  can  leverage 
this  understanding  to  interpret  and  understand  the  model's  results.  For  example,  the  iCAST 
component  of  ICEWS  contains  agent-based  models  (ABMs)  of  specific  countries  for  specified  EOls. 
iCAST  calibrates  these  ABMs  to  the  current  state  of  the  modeled  country  and  then  executes  them 
to  produce  multiple  trajectories  of  potential  future  states  of  that  country.  The  actions  and 
interactions  of  the  agents  in  the  ABM  that  lead  to  each  future  state  are  based  on  rules  and 
environmental  conditions  that  describe  the  country  and  the  sociopolitical  theory  guiding  the  ABM. 
For  example,  all  of  the  ABMs  developed  for  ICEWS  stem  from  a  generic  political  model  that 
captures  rules  based  on  political  theory  for  actions  such  as  protest,  rebellious  activity,  and  religious 
violence.  As  planners  recognize  and  comprehend  the  political  theory  at  play  in  the  ABMs  and  the 
country's  environmental  conditions  framing  the  initial  state  of  the  ABM,  the  model  output  enables 
them  to  understand  the  variety  of  potential  future  states  that  the  model  generates  and  hence 
potential  futures  for  the  country  in  question. 

Within  the  possible  futures,  analysts  may  identify  threats  or  opportunities  that  may  manifest 
themselves,  as  well  as  specific  contextual  factors  that  could  result  from  underlying  operational 
factors.  For  example,  the  model  might  predict  high  levels  of  protest  that  may  indicate  depleted 
resources  or  insurgent  propaganda.  Those  operational  factors  could  suggest  a  potential  COA  for 
mitigation.  Planners  also  benefit  from  the  ability  to  experiment  with  model  input  parameters, 
manipulating  the  model  in  order  to  determine  how  changes  in  input  affect  potential  futures. 
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3.2.2.  Skills  to  understand  COAs  of  other  countries 

Concurrently,  COA  developers  must  consider  what  activities  other  countries  are  engaged  in  or  are 
planning  to  leverage,  exploit,  or  contend  with  that  may  affect  progress  toward  achieving  the 
desired  end  state.  Potential  COAs  of  transnational  actors  or  influential  countries  both  inside  and 
outside  the  AOR  will  favor  distinct,  though  possibly  overlapping  or  competing,  interests  and 
objectives.  Identifying  and  understanding  potential  COAs  of  influential  countries  or  groups  may 
require  the  type  of  knowledge  described  above  for  identifying  relevant  contextual  factors,  and 
definitely  requires  unique  skills  for  identifying  possible  high-level  strategic  agendas  of  other  parties 
that  may  have  an  impact  on  the  strategic  agenda  of  the  CCMD. 

Once  an  analyst  has  characterized  the  interests  of  external  parties,  distinguishing  COAs  for 
transnational  actors  or  for  countries  inside/outside  the  AOR  calls  for  the  recognition  of  high-level 
objectives  that  may  conflict  with  or  otherwise  affect  the  strategic  agenda  of  the  CCMD.  The  process 
for  recognizing  relevant  factors  is  similar  to  the  process  for  identifying  contextual  factors  described 
in  section  3.2.1,  the  process  for  mapping  contextual  factors  to  high-level  objectives  described  in 
section  3.2.3,  and  the  process  of  translating  objectives  to  COAs  described  in  section  3.2.4. 

Once  analysts  have  noted  the  contextual  factors  relevant  to  the  CCMD's  strategic  agenda,  they 
need  the  skill  to  leverage  the  model's  capability  to  explore  external  factors  that  may  have  an 
impact  on  the  contextual  factors  associated  with  the  strategic  agenda.  After  identifying  these 
external  factors,  analysts  must  consider  the  interests  of  other  parties  in  these  external  factors. 
Scenarios  illustrating  how  the  interests  of  transnational  actors  or  other  countries  can  affect 
external  factors  result  in  identification  of  third  party  agendas  that  could  threaten  or  present 
opportunities  for  the  CCMD's  strategic  agenda. 

3.2.3  Skills  to  identify  objectives 

Planners  must  decompose  high-level  strategic  guidance  into  tractable  objectives  that  can  be 
aggregated  to  realize  the  desired  end  state.  Identifying  these  objectives  and  the  assumptions  that 
may  underlie  them  is  critical  for  developing  valid  COAs.  HSCB-relevant  skills  must  enable  planners 
to  translate  strategic  guidance  into  clear  operational  objectives  and  articulate  the  key  planning 
assumptions  within  the  context  of  the  unique  sociocultural  elements  of  the  particular  target 
country/region. 

HSCB  modeling  and  model  results  have  become  increasingly  useful  in  understanding  the 
relationship  between  a  desired  end  state  and  its  relationship  to  the  operational  environment.  For 
example,  the  desired  end  state  of  effective  cooperation  between  the  United  States  and  Country  X 
may  be  achieved  through  COAs  that  support  various  objectives,  including  developing  pro-American 
sentiment  within  the  population. 

3.2.4.  Skills  to  generate  candidate  COAs 

Having  accomplished  the  previous  steps,  COA  developers  must  consider  how  to  relate  specific 
objectives  to  the  possible  COAs,  identify  the  resources  required  for  each  COA,  and  prioritize  the 
COAs.  Consider  the  operational  objective  of  eliminating  the  support  base  of  a  particular  adversary. 
Planners  might  consider  multiple  COAs,  such  as  military  information  support  operations  (MISO), 
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increased  security  in  certain  areas,  kinetic  attacks  on  known  adversary  command  and  control 
elements,  and  non-kinetic  attacks  on  adversary  propaganda  sites.  Current  kinetic  models,  based  on 
the  principles  of  physics  and  on  extensive  information  on  how  systems  operate  (e.g.,  integrated  air 
defense  systems),  can  give  users  confidence  in  the  accuracy  of  their  estimates.  By  contrast,  HSCB 
systems  use  theories  from  areas  of  study  such  as  sociology,  anthropology,  and  psychology  that  are 
much  less  (if  at  all)  grounded  in  physical  principles,  yet  are  still  capable  of  providing  valuable  and 
relevant  insights.  While  most  operational  planners  are  very  familiar  with  kinetic  principles,  their 
understanding  of  social  sciences  is  often  anecdotal  at  best.  As  such,  operational  planners  must 
learn  when  it  is  appropriate  to  apply  HSCB  models,  and  how  to  interpret  the  model  results  with 
respect  to  the  degree  of  confidence  the  model  merits  within  the  context  it  is  being  applied. 

3.3.  Skills  for  COA  Analysis 

In  the  TCP  use  case,  planners  must  model  the  actions  involved  in  a  proposed  COA  to  determine 
how  implementing  those  actions  might  affect  the  operational  environment.  Exploring  how  DIME 
actions  may  affect  the  variables  input  to  a  forecast  model  supports  COA  analysis.  If  a  DIME  action 
can  be  associated  with  a  change  in  a  model's  input  variables,  the  model  will  reveal  how  those 
changes  affect  the  forecast  probability. 

Planners  should  assess  the  feasibility  of  a  COA  to  determine  whether  the  proposed  action  can 
accomplish  the  mission  objective  within  the  allocated  time,  space,  and  resources.  To  support  COA 
feasibility  assessments  planners  need  the  information  and  tools  that  enable  them  to  model  and 
assess  resource  requirements  and  the  impacts  of  DIME  actions  and  others. 

During  the  COA  analysis  phase  planners  must  also  evaluate  the  acceptability  of  a  COA.  This  analysis 
considers  whether  the  benefits  of  the  COA  outweigh  the  expected  costs  in  terms  of  such  factors  as 
losses  of  friendly  forces,  time,  position,  and  opportunity.  Acceptability  analysis  also  ensures  that 
the  COA  accommodates  constraints  such  as  rules  of  engagement  for  specific  operating 
environments. 

Wargaming  offers  an  effective  preliminary  means  of  performing  feasibility  and  acceptability 
assessments.  Wargaming  provides  a  context  to  identify  the  advantages  and  disadvantages  of  a 
COA,  ultimately  allowing  for  a  comparison  among  COAs.  The  wargame  iterates  a  process  of  action, 
reaction,  and  counteraction  that  fosters  ideas  and  insights  that  inform  preliminary  validity 
assessments  of  potential  COAs. 

Wargaming  for  COA  analysis  may  be  entirely  manual  or  computer  assisted.  Both  forms  require 
similar  training  regarding  the  development  of  tactical  judgment  and  operational  experience,  but 
computer-assisted  wargaming  incurs  the  additional  overhead  of  loading  scenarios  into  the  system 
and  training  users  on  the  system.  However,  the  versatility  of  computer  simulations  for  supporting 
multiple  scenarios  mitigates  this  cost. 

To  perform  COA  analysis,  the  operator  must  investigate  the  impact  that  a  particular  action  may 
have  in  a  given  environment.  Models  that  facilitate  "what-if"  investigation— also  known  as 
hypothesis  testing—  of  the  scenario  of  interest  in  the  operational  area  can  assist  this  impact 
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assessment.  For  example,  a  model  could  help  operators  to  answer  questions  such  as  "If  we 
increased  forces  in  this  area,  how  would  the  adversary  or  friendly/neutral  actor  respond?"  or  "If 
the  adversary  took  a  particular  action,  what  would  be  the  most  effective  counteraction?" 

3.4.  Skills  for  COA  Assessment 

COA  assessment  includes  Measures  of  Effectiveness  (MOEs)  for  evaluating  the  accomplishment  of 
the  mission  objectives  and  achievement  of  the  desired  end  state  (CJCSI-3170.01E,  2005).  MOEs 
may  consist  of  more  detailed  measures,  specifically  Measures  of  Performance  (MOPs)  and 
Measures  of  Suitability  (MOSs).  MOPs  detail  in  quantitative  terms  how  well  a  particular  activity  was 
executed;  for  example,  whether  a  port  visit  occurred  or  was  prevented  by  weather  or  other  factors. 
MOSs  relate  to  the  feasibility  of  the  COA  within  the  operational  environment,  reflecting 
operational  readiness  or  availability  with  respect  to  the  intended  action;  for  example,  whether  the 
available  resources  could  support  a  port  visit,  or  whether  elements  of  the  logistical  infrastructure 
could  not  meet  the  resource  requirements.  For  example,  a  MOP  assesses  the  performance  of  a 
port  visit,  an  MOS  assesses  the  operational  capability  to  support  a  port  visit,  and  the  MOE  assesses 
the  impact  of  a  port  visit  on  the  overall  mission  objective  in  terms  of  diplomatic,  show-of-force,  and 
economic  effects. 


4.  Implications  for  Training  for  Mitigation 

Training  for  mitigation  must  lead  to  the  development  of  FISCB-related  expertise.  As  technology 
capabilities  emerge  that  support  the  management  of  FISCB  content  for  application  to  mitigation 
scenarios,  the  training  community  will  learn  more  about  what  training  specific  to  this  context 
requires.  One  concern  is  that  in  order  to  train  individuals  for  a  specific  mitigation  task  using  FISCB 
data,  it  must  be  assumed  that  the  individual  has  been  provided  with  a  degree  of  education.  There  is 
no  clear  documentation  of  what  education  is  required  for  training  in  mitigation.  Flaving  preliminary 
knowledge  and  understanding  of  the  FISCB  context  of  the  deployment  environment  provides  each 
trainee  with  better,  deeper  awareness  of  the  human-related  forces  at  play  in  that  environment  so 
as  to  be  able  to  better  perform  the  task  being  trained.  This  section  distinguishes  between 
education  and  training  expectations  and  describes  recommended  learning  objectives  for  both  with 
respect  to  mitigation.  We  also  describe  emerging  technologies  that  support  such  education  and 
training  or  technological  gaps  related  to  each  learning  objective. 

4.1.  Description  of  the  Mitigation  Training  Space 

Cultural  understanding  doesn't  just  help  you  achieve  your  objectives— it  helps  you  discover 

what  your  objectives  should  be. 

—  General  Anthony  Zinni  (Rasmussen  &  Seick,  2012,  p.  1) 

While  many  breakthroughs  in  FISCB  technology  have  the  promise  of  fundamentally  changing  the 
military  planning  community's  approach  to  mitigation,  military  members  will  need  to  be  trained  on 
how  to  use  the  rapidly  growing  amount  of  information  and  new  tools  before  the  promise  of  better 
mitigation  can  be  reached,  historically,  formal  U.S.  military  training  for  crisis  mitigation  based  on 
cultural  factors  has  focused  on  potential  nation-state  level  reactions  to  blue  courses  of  action. 
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More  recently,  mitigation  training  has  taken  a  strong  tactical  focus  involving  individual  soldier 
training  in  face-to-face  cross-cultural  contact.  The  tools  and  cultural  information/understanding  did 
not  exist  in  the  military  to  conduct  operational-level  mitigation  training.  Training  articles  are 
beginning  to  discuss  an  expanded  view  of  the  cultural  training,  to  include  such  concepts  as 
''culture-general  capabilities"  and  "cross-cultural  competence"  (3C),  but  these  ideas  still  focus 
primarily  on  mitigating  tactical  level  activity  that  might  cause  unanticipated  strategic 
consequences.  (Rasmussen,  Sieck,  &  Osland,  2010;  Wisecarver,  Ferro,  Foldes,  Adis,  Hope,  &  Hill, 
2012) 

Even  with  a  tactical  training  view,  work  remains  to  be  done  on  defining  the  best  approaches  to 
improving  the  military's  cultural  understanding,  and  applying  that  understanding  to  mitigate 
undesirable  outcomes.  "Current  efforts  towards  defining  and  scoping  culture-general  capabilities, 
or  Cross-Cultural  Competence  (3C)  include  high  level  cognitive  skills  such  as  sensemaking  and 
perspective  taking  (Abbe,  Gulick,  &  Herman,  2007)— however,  the  field  has  yet  to  effectively 
characterize  the  cognitive  processes  that  these  skills  entail."  (Rasmussen,  Sieck,  &  Osland,  2010,  p. 
2) 

Though  training  objectives  don't  currently  exist  for  the  new  portfolio  of  HSCB  tools,  each  of  the 
services  has  developed  training  for  individual  cultural  understanding.  For  example,  the  Marine 
Corps  Operational  Culture  And  Language  Training  And  Readiness  (T&R)  Manual  (p.  21),  training 
objectives  include: 

•  Assess  the  attitudes  among  a  foreign  populace 

•  Assess  the  behaviors  among  a  foreign  populace 

•  Assess  cultural  considerations  that  affect  the  population's  attitudes/behaviors 

•  Incorporate  cultural  considerations  into  plans  and  operations 

•  Develop  TTPs 

•  Implement  plans  to  target  the  desired  attitudes/behaviors 

•  Monitor  the  effectiveness  of  plans  targeting  attitudes/behaviors 

•  Reassess  the  population's  attitudes/behaviors 

•  Adjust  operations 


These  training  objectives  are  representative  of  how  the  services  are  training  their  planners  and 
operators  to  consider  sociocultural  factors.  These  objectives  are  often  taught  as  thought  exercises, 
and  rarely  extend  past  an  initial  understanding  of  potential  first  order  effects.  Some  more  complex 
capabilities,  such  as  red  teaming,  are  taught  to  limited  groups  and  begin  to  train  military  personnel 
on  how  adversaries  are  likely  to  respond  to  blue  courses  of  action.  The  objective  of  red  teaming  is 
to  "avoid  groupthink,  mirror  imaging,  cultural  missteps,  and  tunnel  vision  in  plans  and  operations" 
(University  of  Foreign  Military  and  Cultural  Studies,  2011,  p.  1).  These  techniques  provide  great 
value  to  operational  planning,  but  seldom  extend  beyond  considering  the  stated  adversary  into 
considering  the  reaction  of  the  surrounding  population. 

Innovations  such  as  those  described  here  are  designed  to  support  operational  level  tasks  such  as 
course  of  action  generation,  etc.  Today's  HSCB  tools  bring  the  capability  of  not  only  considering  the 
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adversary,  but  of  multiple  facets  of  the  local  population.  This  will  radically  increase  the  number  of 
elements  of  mitigation  that  can  be  considered  by  planners,  if  the  tools  are  made  more  widely 
available  and  properly  trained.  Training  will  need  to  include  not  only  formal  courses,  but  also 
integration  into  exercises  and  Master  Scenario  Event  Lists  (MSELs)  and  exercises. 

4.2.  Mitigation  Education  and  Training:  Bridging  the  Gap 

Before  we  can  expect  a  trainee  to  be  able  to  perform  a  mitigation  task  which  in  the  real  world  must 
be  performed  in  the  face  of  uncertainty,  we  must  first  educate  the  trainee  to  better  understand 
and  act  in  an  uncertain  space.  As  a  result,  training  for  mitigation  must  be  preceded  by  education. 
To  develop  education  curricula  to  support  mitigation  training,  "we  must  cultivate  an  environment 
in  which  we  teach  our  soldiers  how  to  think  and  adapt,  not  what  to  think."  (Burton,  Nance,  & 
Walton,  2011,  p.  1)  In  other  words,  mitigation  education  must  build  "a  culture  of  flexibility"  in 
thinking  to  support  the  training  of  specific  mitigation  tasks  (Burton,  Nance,  &  Walton,  2011). 

For  example,  in  order  to  effectively  mitigate  scenarios  dealing  with  information  about  events  or 
actions  taking  place  in  a  foreign  environment,  the  user  of  the  system  must  have  received  education 
previously.  In  particular,  the  kind  of  education  relevant  to  such  a  context  may  be  education  in  the 
norms  and  traditions  of  the  associated  culture,  as  well  as  education  in  the  understanding  and 
awareness  of  cognitive  biases.  This  education  background  prepares  the  user  to  be  able  to  mitigate 
uncertainties  that  may  arise  from  being  informed  of  events  or  actions  contrary  to  what  the  user  is 
accustomed  to  in  his/her  personal  cultural  experiences. 

In  this  section,  we  will  identify  learning  objectives  for  education  for  mitigation.  Strategies  that  have 
been  employed  for  achieving  such  objectives  are  identified.  As  tools  to  support  mitigation  have 
been  developed,  the  question  of  how  much  education  is  needed  to  support  use  of  the  systems  is 
an  open  question.  If  too  much  education  is  required,  then  the  system  requires  over-specialization 
of  its  users,  and  cannot  be  sustained.  For  example,  if  a  user  needs  to  have  a  PhD  in  statistics  to  be 
able  to  interpret  results  presented  by  these  systems  or  to  understand  the  significance  of  events  in  a 
data  set,  then  the  system  will  not  be  able  to  be  effectively  employed  by  the  target  users.  If  the  user 
is  unable  to  understand  what  the  system  is  doing  or  how  it  works,  then  the  lack  of  confidence  in 
the  results  will  undermine  the  significance  of  the  content.  For  example,  if  a  mitigation  support 
system  is  indicating  the  decreased  likelihood  of  a  violent  event,  but  the  system  is  not  able  to 
communicate  to  a  layperson  user  what  factors  are  leading  to  the  assessment,  then  the  user  is  less 
likely  to  trust  the  analysis  and  hence  less  likely  to  feel  comfortable  leveraging  the  information  for 
building  COA.  Ongoing  work  is  attempting  to  exploit  visualization  and  automation  techniques  to 
support  verification  and  validation  of  models  and  data  to  lower  the  education  threshold  for  using 
these  systems.  For  example,  the  Model  Evaluation  Selection  and  Assessment  (MESA)  project  under 
the  Office  of  Naval  Research  is  investigating  innovative  verification  and  validation  techniques  for 
HSCB  models  to  support  the  transparent  and  accessible  understanding  of  how  HSCB  models  work, 
the  data  they  use  and  produce,  and  the  theoretical  principles  that  drive  the  model  (Ruvinsky, 
Wedgwood,  &  Welsh,  2012). 
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This  section  describes  recommended  objectives  for  education  and  training  for  mitigation,  as  well  as 
emerging  technologies  that  support  such  training  or  technological  gaps  related  to  each  learning 
objective. 

4.2.1.  Education  learning  objective  #1:  Understanding  human  dynamics 

Sociocultural  sensemaking  for  mitigation  requires  understanding  ''human  dynamics/'  defined  as 
"the  actions  and  interactions  of  personal,  interpersonal,  and  social/contextual  factors  and  their 
effects  on  behavioral  outcomes"  (Defense  Science  Board,  2009,  p.  vii).  In  the  HSCB  domain,  human 
dynamics  propel  change  over  time.  Factors  that  influence  human  dynamics  include  economics, 
religion,  politics,  and  culture.  Developing  skills  in  understanding  human  dynamics  entails 
acquisition  of  two  distinct,  yet  complementary,  perspectives:  a  culture-general  perspective  on 
human  behavioral  tendencies,  and  a  culture-specific  perspective  in  which  knowledge  of  these 
general  behavioral  tendencies  is  applied  to  a  specific  cultural  context  to  develop  an  understanding 
of  a  particular  culture. 

Human  Behavior  Models  (HBM)  are  used  within  simulations  for  training  in  which  the  task  being 
trained  requires  understanding  of  other  people,  either  individually  (e.g.,  human  psychology)  or  in  a 
population  (e.g.,  sociology),  for  decision  making.  They  provide  variations  in  human  behavior  in 
order  to  produce  a  more  realistic  experience  for  the  trainee  (Wray  &  Laird;  2003). 

The  use  of  HBM  to  establish  the  background  context  to  training  simulations  makes  it  clear  that 
education  of  the  dynamics  of  human  behavior  is  a  pre-requisite  to  such  training  scenarios  including 
scenarios  relevant  to  mitigation  where  decisions  are  being  made  that  impact  populations. 

4.2.1. 1.  Education  for  human  behavior  awareness 

Education  to  foster  awareness  regarding  human  behavior  must  familiarize  the  trainee  with  the 
cultural  factors  that  influence  decision  making  at  the  micro  level,  as  well  as  the  kind  of  macro-level 
behaviors  that  emerge  from  these  micro-factors.  For  example,  ethnic  diversity  is  the  property  of  a 
geographical  region  and  is  described  by  having  a  population  in  which  wide  variety  of  races  are 
found.  In  this  example,  the  racial  status  of  the  individual  contributes  to  the  ethnic  diversity  of  an 
area,  which  in  turn  impacts  aspects  of  the  environment  which  influences  an  individual's 
experience.  According  to  Kanbur,  Rajaram,  &  Varshney  (2011,  p.  149),  "The  daily  life  of  families, 
neighborhoods,  regions  and  countries  is  influenced  by  ethnic  diversity."  Various  research  projects 
hypothesize  a  positive  association  between  ethno-linguistic  fractionalization  index  (ELF;  a  measure 
of  ethnic  diversity)  and  the  probability  of  civil  war  (Collier,  Elliott,  Hegre,  Hoeffler,  Reynal-Querol,  & 
Sambanis,  2003).  Education  for  human  behavior  awareness  must  leverage  various  disciplines, 
including  psychology,  sociology  and  anthropology,  to  name  just  a  few. 

Computational  social  science  (CSS)  models  and  HSCB  data  work  to  support  an  understanding  of 
culture-general  processes  of  human  dynamics.  Worldwide  ICEWS  (W-ICEWS)  has  a  modeling  and 
forecasting  component  called  iCAST  that  leverages  CSS  models  and  HSCB  data  to  predict  events  of 
interest.  Though  shown  to  be  useful  and  produce  interesting  and  relevant  content,  a  significant 
challenge  to  this  forecasting  capability  is  the  presentation  of  the  sophisticated  computational 
models  and  analyses  to  laypeople.  To  promote  confidence  in  the  iCAST  models  and  forecasts,  the 
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development  team  built  a  transparency  capability  that  allows  the  user  to  explore  the  factors 
considered  by  the  models  and  the  relationships  between  the  factors  and  the  forecasted  results. 
The  transparency  capability  not  only  contributes  to  the  user's  awareness  and  understanding  of  how 
the  model  works  by  exposing  the  variables  used  by  the  models  and  allowing  the  user  to  change  the 
values  of  variables  to  explore  "what-if"  scenarios,  but  also  provides  insight  into  the  sociological 
implications  of  the  model.  Recent  versions  of  iCAST  leverage  elements  of  the  MESA  program 
capabilities  (Ruvinsky,  Wedgwood,  &  Welsh,  2012),  which  provide  techniques  and  tools  to  support 
the  verification  and  validation  of  HSCB  models. 

4.2.1. 2.  Education  for  cultural  awareness 

Cultural  Awareness  as  Cross-Cultural  Competency  (3C)  is  defined  by  Selmeski  (2007,  p.  12)  as  "the 
ability  to  quickly  and  accurately  comprehend,  then  appropriately  and  effectively  engage  individuals 
from  distinct  cultural  backgrounds  to  achieve  the  desired  effect,  despite  not  having  an  in-depth 
knowledge  of  the  other  culture."  This  competency  goes  beyond  developing  verbal  communication 
skills  for  interacting  with  a  particular  culture,  and  toward  developing  culture-general  knowledge  as 
a  framework  for  interacting  with  many  different  cultures.  Understanding  how  culture  influences 
human  behavior  is  a  first  step  toward  understanding  aspects  of  human  dynamics  in  general,  but 
real-world  contexts  are  always  culture  specific.  For  example,  the  culture-general  recognition  that 
norms,  beliefs,  and  customs  influence  the  behaviors  and  interactions  of  individuals,  groups,  and 
societies  helps  analysts  to  identify  or  instantiate  the  norms,  beliefs,  and  customs  that  prevail  in  a 
particular  culture  of  interest.  Analysts  can  then  begin  to  reason  about  the  impact  of  these 
instantiations  on  the  behaviors  and  interactions  of  the  individuals,  groups,  and  societies  in  that 
culture. 

Various  technologies  are  being  developed  to  support  training  in  culture-specific  awareness.  For 
example,  simulation  environments  for  cultural  training  such  as  the  Virtual  Cultural  Awareness 
Trainer  (VCAT)  have  been  shown  to  improve  cultural-specific  awareness  (Johnson,  Friedland, 
Schrider,  Valente,  &  Sheridan,  2011).  In  fact,  U.S.  Southern  Command  has  designated  the  VCAT  as 
"Mandatory  Training"  for  various  culture-specific  exercises  (Alelo  -  Evidence  of  Effectiveness, 
2013). 

4.2.2.  Education  learning  objective  #2:  Awareness  and  diminishing  of  cognitive  bias 

The  term  "cognitive  bias"  refers  to  subconscious  errors  that  humans  make  in  processing 
information.  Such  bias  can  lead  to  irrational  or  illogical  actions.  Trainees  must  learn  to  recognize 
cognitive  bias  in  two  contexts:  (1)  how  cognitive  biases  affect  the  interaction  of  populations  within 
the  AOR,  and  (2)  the  trainee's  own  susceptibility  to  such  biases  as  they  evaluate  potential  COAs. 
Becoming  aware  of  these  cognitive  vulnerabilities  is  a  first  step  toward  mitigating  them. 

4.2.2. 1.  Education  to  recognize  evaluation  biases 

Understanding  the  benefits  and  limitations  of  data  sources  used  by  HSCB  tools  helps  planners  to 
evaluate  the  evidence  presented  by  these  tools.  People  are  susceptible  to  a  variety  of  extraneous 
influences  when  determining  the  evidence  to  rely  on  and  how  to  interpret  that  evidence. 
Significant  cognitive  biases  that  can  arise  when  evaluating  evidence  include  confirmation  bias,  in 
which  people  place  disproportionately  great  reliance  on  information  that  supports  their  own 
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beliefs  or  conclusions.  HSCB  data  often  describes  aspects  of  human  existence  about  which  all 
people  have  ideas,  opinions,  and  guiding  principles  or  hypotheses.  Analysts  must  be  very  careful  to 
weigh  the  data  objectively,  and  not  to  succumb  to  "the  natural  tendency  of  human  beings  to  see 
what  they  expect  to  see"  (Reese,  2012,  p.  1261). 

People  are  also  subject  to  vividness  bias,  in  which  compelling,  concrete,  and  personal  information 
has  a  greater  impact  on  the  evaluation  of  a  situation  than  more  relevant,  but  abstract,  information 
such  as  statistics.  Vividness  bias  plays  a  role  in  assessing  HSCB  data  because  information  garnered 
from  HSCB  sources  such  as  social  media  often  communicates  memorable  personal  experiences  in 
emotionally  charged  language  (Heuer,  1999).  Another  form  of  bias,  the  anchoring  effect,  means 
that  "focusing  attention  on  a  single  option  may  lead  the  decision  maker  to  neglect  potentially 
valuable  alternatives"  (Missier,  Ferrante,  &  Costantini,  2007).  Planners  must  give  consideration  to 
all  relevant  and  viable  alternatives  in  the  context  of  a  decision  problem. 

As  Keren  (1990)  posits,  there  are  "cognitive  pills  for  cognitive  ills."  The  term  "debiasing"  describes 
any  technique  used  to  prevent  or  mitigate  cognitive  biases.  These  techniques  include  evidence 
lineup,  which  may  mitigate  the  anchoring  effect  by  specifying  a  "lineup"  of  possible  alternatives  to 
consider,  and  competitive  self-regulation,  in  which  analysts  compete  with  each  other  to  produce 
analytical  products  free  from  cognitive  biases.  An  emerging  technology  designed  to  identify  and 
mitigate  cognitive  biases  in  analysts  comes  from  the  Intelligence  Advanced  Research  Projects 
Activity's  (lARPA's)  SIRIUS  program,  which  uses  Serious  Games  techniques  and  technologies  to  train 
participants  to  recognize  and  mitigate  various  cognitive  biases. 

4.2.2.2.  Training  to  recognize  cause  and  effect  biases 

When  using  HSCB  data  and  models  to  perform  COA  mitigation,  analysts  must  make  judgments 
about  cause  and  effect  as  they  identify  possible  COAs  that  would  lead  to  the  desired  end  state, 
analyze  COAs  to  understand  how  causal  factors  impact  the  current  state,  and  assess  the  utility, 
viability,  and  affordability  of  each  COA.  Perceptions  about  cause  and  effect  can  be  influenced  by 
centralized  direction  bias,  which  leads  people  to  attribute  effects  to  coordinated  actions  or 
conspiracies  rather  than  to  accidents,  mistakes,  or  coincidence.  In  addition,  analysts  may  assume 
that  causes  and  effects  are  proportional  to  each  other.  In  other  words,  large  effects  must  have 
large  causes.  The  training  debiasing  techniques  described  in  section  4.2.1,  such  as  using  lineups, 
can  help  trainees  to  counter  cause  and  effect  biases. 

4.2.3.  Training  learning  objective  #1:  Develop  assessments  of  HSCB  dimensions  of  foreign 
populace 

The  first  training  learning  objective  one  needs  to  achieve  in  order  to  perform  mitigation  is  to 
explore  the  data  describing  the  foreign  populace  in  question.  As  an  example  of  the  kind  of  training 
that  supports  this  objective,  we  will  present  the  iTRACE  and  iCAST  components  of  the  W-ICEWS 
suite,  and  describe  the  training  initiatives  conducted  for  these  components.  ICEWS  was  designed  to 
forecast  instability  events  for  countries  in  an  AOR.  ICEWS  consists  of  various  components  designed 
to  support  the  visualization  and  understanding  of  data  and  forecasts  generated  by  the  ICEWS 
system.  ICEWS  is  composed  of  three  modules:  iTRACE,  iCAST  and  iSENT.  Both  iTRACE  (ICEWS 
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Trending,  Recognition  &  Assessment  of  Current  Events)  and  iCAST  (ICEWS  Forecasting)  have  been 
demonstrated  and  trained  to  operational  users. 

4.2.3. 1.  Task  1:  Identify,  explore  and  analyze  data  relevant  aspects  ofHSCB  perspectives  of  a 
foreign  populace  to  be  investigated  with  respect  to  a  mission  objective. 

The  primary  goal  of  iTRACE  is  to  provide  a  fully  automated  capability  to  monitor  political  activity 
around  the  globe.  This  is  accomplished  by  automatically  converting  news  reports  into  structured 
indices  that  reflect  the  character  and  intensity  of  interactions  between  key  leaders,  organizations, 
and  countries.  The  iTRACE  system  provides  a  data-driven  view  into  who  is  doing  what  to  whom , 
when ,  where  and  how.  The  system  is  based  on  open-source  reporting  from  a  large  number  of  major 
and  regional  news  services.  iTRACE  processes  these  news  reports  to  extract  the  character  and 
intensity  of  interactions  between  entities  such  as  leaders,  organizations  and  countries.  The 
processed  content  may  then  be  used  by  computational  models  to  discover,  identify  and  explore 
trends  and  patterns,  and  ultimately  indicate  potential  of  impending  violence,  crises  or  conflicts. 

The  training  goal  was  to  prepare  the  operational  users  at  USPACOM  to  use  the  iTRACE  system  to 
meet  the  command's  needs  for  HSCB  information,  such  as  providing  information  on  event  trends  in 
countries  of  interest  or  interaction  trends  between  countries  or  actors  over  time.  Many  of  the 
learning  objectives  of  the  training  dealt  with  being  able  to  use  the  iTRACE  system  itself,  such  as 
navigating  and  using  the  iTRACE  interface,  but  there  were  learning  objectives  designed  to  teach 
users  how  to  aggregate  content  and  generate  visualizations  of  content  that  support  a  particular 
HSCB  informational  need.  It  is  important  to  note  that  the  iTRACE  training  packet  assumed  that  the 
users  had  previous  education  regarding  aspects  of  the  human,  social,  cultural  and  behavioral 
domains  being  investigated,  such  as  education  in  human  dynamics,  both  general  and  specific  to  the 
region  in  question,  giving  the  user  awareness  and  understanding  of  what  HSCB  information  is 
relevant  to  the  mission  context.  This  education  allows  the  user  to  identify  aspects  of  the 
investigation  that  he  or  she  will  use  to  configure  the  iTRACE  analysis,  such  as  what  cultural  or 
religious  groups  are  of  interest?  What  leaders  or  organizations  are  relevant?  What  kinds  of  events 
or  intensity  of  events  are  most  interesting? 

The  iTRACE  training  was  broken  down  into  three  scenarios.  The  first  consisted  of  a  demonstration 
of  the  system;  the  second  allowed  the  trainees  to  reproduce  the  analysis  that  we  demonstrated  in 
the  first  scenario;  and  the  third  scenario  posed  questions  to  the  trainees  for  which  they  used  the 
iTRACE  system  to  answer.  These  questions  were  decomposed  into  three  categories  in  which  the 
trainees  were  to  investigate  (1)  activities  between  actors  (e.g.,  what  activities  in  the  last  10  years 
involved  Philippines  towards  the  United  States?),  (2)  event  filtering  (e.g.,  find  events  that  involved 
Philippine  military  activity  toward  insurgency  groups  between  Jan  2001  and  Oct  2009),  and  (3) 
actor  filtering  (e.g.,  determine  what  recent  events  involved  the  president  Gloria  Arroyo  of  the 
Philippines). 

4.2.3.2.  Task  2:  develop  relevant,  data-driven  assessments  based  on  HSCB  data  and  tools  that 
capture  aspects  of  a  foreign  populace  relevant  to  CO  A  development 

iCAST  provides  forecasts  of  the  likelihood  of  the  occurrence  of  events  of  interests  (EOls),  namely 
domestic  political  crisis,  ethnic/religious  violence,  insurgency,  international  crisis  and  rebellion  and 
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the  key  factors  driving  them.  The  primary  objective  of  iCAST  is  to  provide  human,  social,  cultural 
and  behavioral  awareness  to  the  user  regarding  a  forecasted  country  state.  This  is  accomplished  by 
aggregating  the  forecasts  of  multiple  independent  HSCB  models  to  create  a  single,  aggregate 
forecast  that  is  more  accurate  that  any  one  single  model.  By  integrating  improved  versions  of  best 
of  breed  models  from  multiple  perspectives,  iCAST  achieves  more  accurate,  precise  forecasts  than 
any  one  model  alone. 

The  iCAST  training  packet  had  four  training  objectives:  providing  the  trainee  with  the  ability  to 

•  View  forecasts  for  different  countries  and  EOls  (e.g.,  what  is  the  anticipated  probability  of  a 
specific  EOI  for  a  particular  country  at  a  future  point  in  time?) 

•  Analyze  aggregate  model  forecasts  with  respect  to  individual  model  contributions  (e.g., 
how  accurate  are  the  models  that  are  contributing  to  the  aggregated  forecast?) 

•  Analyze  specific  model  forecasts  with  respect  to  underlying  data  parameter  contributions 
(e.g.,  what  are  the  key  factors  that  contribute  to  the  likelihood  of  a  particular  EOI  occurring, 
or  not  occurring,  for  some  country?) 

•  Explore  data  parameter  sensitivity  within  a  specific  model  forecast  (e.g.,  what  might 
influence  the  probability  of  EOI  occurring,  or  not  occurring,  for  a  particular  country?) 

iCAST  training  guided  trainees  through  the  navigation  and  use  of  the  various  visualizations  and 
content  provided  in  iCAST  to  answer  questions  such  as  those  presented  above. 

Once  data  has  been  explored  and  a  knowledge  and  awareness  of  the  space  has  been  established, 
courses  of  action  may  be  developed  to  realize  mission  objectives  while  regarding  the  human, 
social,  cultural  and  behavioral  dimensions  of  the  operational  environment. 

4.2.4.  Training  learning  objective  #2:  Red  teaming 

The  process  of  red  teaming  provides  alternative  options  to  a  decision.  By  generating  alternatives, 
potential  improvements  in  decision  making  are  exposed.  According  to  University  of  Foreign  Military 
and  Cultural  Studies  (UFMCS,  2011,  p.  1),  "Red  teaming  is  a  structured,  iterative  process,  executed 
by  highly  trained,  educated,  and  practiced  team  members  that  provides  commanders  an 
independent  capability  to  fully  explore  alternatives  to  plans,  operations,  concepts,  organizations, 
and  capabilities  in  the  context  of  the  operational  environment."  Red  teaming  supports  the  decision 
making  personnel  in  seeing  the  operational  environment  from  varying  perspectives  including  that 
of  adversaries  or  coalition  partners. 

4.2.4. 1.  Task:  challenge  CO  As  by  questioning  assumptions,  patterns,  analysis,  etc. 

By  questioning  various  aspects  of  a  COA,  red  teaming  can  generate  important  analytical 
perspectives  that  the  team  may  have  previously  overlooked.  These  perspectives  include  alternative 
analysis,  consideration  of  other  entities  on  the  battlefield,  identification  of  gaps  or  vulnerabilities, 
identification  of  sequences  of  events,  and  identification  of  measures  of  effectiveness.  Red  team 
training  is  conducted  at  UFMCS  at  Fort  Leavenworth.  According  to  U.S.  Army  Training  and  Doctrine 
Command,  (2013,  p.  6),  "The  basis  for  Red  Team  success  relies  on  education  and  training  grounded 
in  theory,  doctrine,  cultural  anthropology,  the  operational  environment  and  focused  Red  Team 
techniques." 
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The  curriculum  at  UFMCS  is  organized  around  five  themes: 

1.  Critical  and  creative  thinking 

2.  Red  Team  techniques  (e.g.,  negotiations,  group  dynamics) 

3.  Understanding  how  culture  impacts  the  operational  environment 

4.  Understanding  the  critical  variables  found  in  the  operational  environment  with  respect  to 
trends  and  interrelationships,  and 

5.  Western,  non-Western  and  non-military  theory 

According  to  the  University  of  Foreign  Military  and  Cultural  Studies  (2011),  there  are  various 
techniques  and  procedures  that  support  the  development  of  red  teaming  capabilities  including  a 
nine  step  cultural  methodology  which  promotes  understanding  of  a  foreign  culture.  The  cultural 
analysis  produced  by  the  nine  steps  provides  a  perspective  known  as  the  "four  ways  of  seeing," 
namely,  "how  X  views  itself,"  "how  Y  views  itself,"  "how  X  views  Y,"  and  "how  Y  views  X."  The  nine 
steps  are: 

1.  Establish  a  base  line  of  understanding  by  examining  the  four  ways  of  seeing 

2.  What  defines  the  Social  System?  (roles  of  family/tribe  and  ways  of  ascribing/acquiring 
status) 

3.  What  are  the  sources  of  power? 

4.  What  are  the  critical  narratives  of  the  cultural  history? 

5.  What  is  the  role  of  the  formal  and  informal  economy? 

6.  What  cultural  forms  and  semiotics  are  endemic  to  the  society? 

7.  What  sociolinguistics  are  evident? 

8.  What  are  their  core  emotional  beliefs? 

9.  In  what  ways  does  the  collected  data  shape  how  Y  thinks? 

Red  Team  courses  are  taught  at  UFMCS  as  seminars  taught  by  academic  and  subject-matter 
experts.  Training  includes  case  studies  for  developing  "best  practices"  capabilities. 


5.  Conclusion 

Emerging  technologies  relevant  to  mitigation  have  the  potential  to  improve  planning,  but  require 
significant  enhancements  to  current  operational  training.  The  knowledge  needed  to  perform 
mitigation  spans  human  dynamics,  DIME  actions  and  their  potential  impact  on  population-centric 
objectives,  and  ways  to  counter  cognitive  biases.  To  develop  the  necessary  skills  and  knowledge, 
analysts  and  planners  should  receive  training  in  human  behavior  awareness  (a  culture-general 
approach)  and  cultural  awareness  (a  culture-specific  approach).  Training  should  also  enable  users 
to  assess  the  impact  of  DIME  actions  on  the  operational  environment  and  human  terrain,  discern 
threats  and  opportunities  associated  with  the  operational  environment,  identify  planning 
assumptions,  and  recognize  and  minimize  various  biases  in  processing  information. 

Current  science  and  technology  can  support  training  in  these  skills.  Still,  many  training  gaps  exist, 
and  the  continued  development  and  fielding  of  HSCB  mitigation  capabilities  make  innovations  in 
training  of  the  skills  and  knowledge  needed  to  perform  COA  mitigation  tasks  imperative. 
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